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Dear  Mr.  Rhorer  and  Mr.  Lam: 

The  Controller’s  Office,  City  Services  Auditor  presents  its  audit  report  concerning  the 
granting  functions  of  the  Human  Services  Agency’s  Department  of  Aging  and  Adult 
Services  (DAAS).  The  audit  objectives  were  to  determine  whether  DAAS  is  effectively 
managing  the  process  of  providing  services  to  seniors  and  adults  with  disabilities  through 
granting  Federal,  State  and  local  funds  to  community  and  non-profit  organizations.  The 
major  audit  findings  include: 

• The  methods  by  which  DAAS  researches  and  determines  the  needs  of  seniors  and 
adults  with  disabilities  in  San  Francisco  meet  State  requirements,  but  the  analysis 
does  not  identify  specific  needs  and  target  populations,  existing  resources  and  gaps 
in  service,  establish  priorities,  set  goals  and  objectives  or  allow  the  Department  to 
allocate  resources  effectively  (Finding  1-A).  We  note  that  the  Human  Services 
Agency  is  currently  updating  this  research  and  analysis  effort. 

• Significant  disparities  exist  between  what  the  needs  assessment  does  show — that 
housing,  transportation  and  in-home  services  are  the  services  most  needed  by 
seniors,  and  what  the  Office  on  Aging  (OOA)  funds — meals,  community  and 
referral  services.  We  note  that  the  Agency  does  fund  housing,  transportation  and 
in-home  services  through  other  programs  (Finding  1-B). 

• The  Department  must  match  Federal  and  State  funds  for  specific  types  of  senior 
services  with  approximately  SI. 8 million  of  local  dollars.  An  estimated  S10  million 
in  discretionary  local  funds  could  be  re-allocated  to  other  services  for  seniors  and 
adults  with  disabilities  and  still  meet  matching  requirements  (Finding  1-C). 

• The  management  system  with  which  the  Department  monitors  contractor 
performance  has  multiple  weaknesses.  Some  providers  administer  their  own 
assessments,  the  tools  for  monitoring  are  inconsistent  and  overly  general,  the 
system  emphasizes  compliance  and  has  little  measurement  of  service  quality,  and 
staff  are  not  properly  trained  or  rotated  (Findings  2- A,  B and  D). 
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• The  database  system  used  by  DAAS  to  monitor  the  units  of  sendee  provided  by 
its  contractors,  SF  GetCare,  is  often  inaccurate  and  unreliable,  making  it 
difficult  to  determine  if  contract  obligations  are  being  met,  and  to  evaluate 
contractor  performance.  We  note  that  the  Department  is  currently  moving  this 
function  to  a combination  of  a new  contractor  and  in-house  staff  but  recommend 
that  the  Department  also  review  its  arrangements  with  the  existing  contractor 
with  whom  it  has  spent  approximately  $2.6  million  from  July  2000  to  the 
present  (Finding  2-C). 

• Contract  awards  made  by  the  Department  and  by  the  Commission  on  Aging 
violated  the  rules  and  guidelines  set  up  in  the  Request  for  Proposals  (RFP) 
process  and  did  not  use  consistent  or  objective  criteria.  Sixteen  contractors  that 
scored  below  the  minimum  point  requirement  were  awarded  funds,  and  a 
variety  of  other  awards  were  made  that  were  not  reflective  of  the  RFP  scores 
and  rankings.  One  contractor  selected  in  this  manner  was  later  de-funded  due  to 
non-performance  of  their  contract  (Findings  3-A,  B and  C). 

• The  OOA  spends  a significant  amount  of  its  budget,  more  than  $1.6  million,  on 
information  and  referral  services — funding  ten  Neighborhood  Resource  Centers 
managed  through  three  non-profit  agencies,  contracting  with  other  agencies  for 
information  and  referral  elements  as  part  of  their  services,  and  running  a unit  in 
the  Department  itself  with  this  responsibility.  There  are  no  common  definitions 
of  units  of  service  and  measurement  approaches  vary  among  these  providers, 
with  the  result  that  clients  are  counted  differently  and  the  Department  does  not 
know  the  outcomes  for  seniors  or  the  effectiveness  of  the  information  and 
referral  services  (Finding  4-A). 

The  overall  audit  conclusions  are  that  improvements  are  needed  in  the  way  the  Department 
of  Aging  and  Adult  Services  researches  the  needs  of  seniors  and  adults  with  disabilities, 
prioritizes  and  allocates  it  funding,  conducts  its  grant  award  processes  and  monitors  the 
performance  of  its  community-based  contractors.  Improvements  in  these  areas  will  mean 
that  services  reach  those  in  most  need,  grant  funds  are  awarded  objectively  to  agencies  that 
are  able  to  deliver  high  quality  services,  and  the  Department  effectively  manages  the 
quantity  and  quality  of  the  services  provided  to  seniors  and  adults  with  disabilities. 

The  Department’s  response  to  the  audit  concurred  with  many  of  the  findings  and 
recommendations  and  is  attached  as  Appendix  X of  this  report. 

We  acknowledge  the  assistance  and  cooperation  provided  to  the  audit  staff  by  the 
Department  and  by  the  service  providers.  The  Controller’s  Office  will  work  with  the 
Department  to  follow  up  on  the  recommendations  made  in  this  report. 

Respectfully  submitted, 


Ed  Hamilton 
Controller 
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EXECUTIVE  SUMMARY 


FINDINGS  AND  RECOMMENDATIONS 


FINDING  1-A.  NEEDS  ASSESSMENT 

The  Department  of  Aging  and  Adults  Services 
(DAAS)  needs  assessment  identifies  broad  categories 
of  need  for  seniors  and  adults  with  disabilities  in  San 
Francisco  but  the  analysis  does  not  identify  specific 
populations  in  need,  their  locations,  the  extent  or  type 
of  need,  or  the  methods  to  meet  them.  The  State- 
mandated  Area  Plan  on  Aging  lists  local  resources  for 
seniors,  but  does  not  include  analysis  of  the  gaps 
between  needs  and  resources.  The  needs  assessment 
does  not  effectively  prioritize  needs  or  target 
populations.  The  overall  effect  of  these  weaknesses  is 
that  the  needs  assessment  is  not  sufficient  to  set  goals 
and  objectives  or  effectively  direct  funding. 

The  audit  recommends  that  the  Department  use  the  needs  assessment  information  to 
identify  specific  needs  and  target  populations,  identify  current  resources  and  gaps  in 
service,  establish  clear  service  priorities,  set  goals  and  objectives,  and  allocate  the  Office 
on  Aging  (OOA)  limited  funds  accordingly.  The  information  should  also  be  used  for 
broader  strategic  purposes  such  as  leveraging  funds  available  from  other  sources  and 
identifying  needed  legislative  changes.  The  Department  may  need  assistance  from  an 
outside  consultant  for  these  purposes  and  should  review  other  county  area  plans  and 
model  its  work  on  some  of  the  best  practice  methodologies  described  in  this  report. 

FINDING  1-B.  FUNDING  ALLOCATION 

The  auditors  examined  the  results  of  the  Area  Plan’s 
needs  assessment  studies  and  took  a tally  of  the  number 
of  times  specific  needs  were  identified.  The  results 
indicate  that  housing,  transportation,  and  in-home  support 
services  are  the  most  identified  needs  for  seniors  in  San 
Francisco,  but  these  receive  only  1%,  4%,  and  2%,  of  the 
OOA  funding,  respectively.  The  OOA  spends  46%  of  its 
funds  on  meal  programs  despite  the  fact  that  meal 
services  were  not  identified  as  the  top  priority  in  the  needs  assessment  process.  State  and 
Federal  funds  specified  for  meals  dictate  around  $4.1  million  of  this  spending,  however, 
the  OOA  also  allocates  $3.9  million  of  its  own  discretionary  local  funds  to  meals 
programs.  The  OOA  also  spends  more  money  on  Information,  Referral  and  Assistance 
services  (estimated  $1.6  million)  than  on  any  direct  service,  except  for  meals  and 
community  services,  despite  the  fact  that  these  services  are  not  identified  as  a priority  in 
the  needs  assessment.  While  the  City  is  spending  hundreds  of  millions  of  dollars  on 


The  OOA’s  funding 
distribution  to  specific 
services  does  not  reflect 
the  needs  most  often 
identified  by  the 
Department’s  own  needs 
assessment  activities. 


The  Department  conducts 
state-mandated  needs 
assessment  activities. 
However  the  analysis  does 
not  identify  specific  needs 
and  target  populations, 
identify  existing  resources 
and  gaps  in  service, 
establish  priorities,  or  allow 
the  Department  to  allocate 
resources  effectively. 
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housing  and  in-home  support  through  other  programs,  the  OOA’s  analysis  and  allocation 
efforts  do  not  demonstrate  that  these  needs  are  in  fact  being  addressed  outside  it’s  own 
granting  program.  Allocation  of  the  OOA  funds  to  contracted  services  is  based  primarily 
on  prior  years'  allocations  rather  than  on  the  needs  assessment  or  appropriate  gap 
analysis.  As  a result  of  these  disparities  and  the  allocation  approach,  there  is  no 
assurance  that  the  OOA  is  funding  the  most  critical  needs  of  seniors  and  adults  with 
disabilities  in  San  Francisco.  The  Area  Plan  and  the  Department’s  needs  assessment 
activities  do  not  justify  the  current  allocation  of  the  OOA  resources  among  the  29 
different  services  provided. 

The  audit  recommends  that  the  Department  link  the  level  of  funding  allocated  to  its 
contracted  services  to  an  improved  needs  assessment.  Specifically,  the  Department’s 
Area  Plan  should  describe  how  the  specific  needs  identified  will  be  addressed  by  the 
services  the  OOA  purchases.  The  allocation  should  clearly  show  how  the  OOA  is  using 
restricted  State  and  Federal  funds  and  local  funds. 

FINDING  1-C.  LOCAL  DISCRETIONARY  FUNDS 

The  OOA  is  required  to  match  State  and  Federal  funds 
for  programs  with  an  estimated  $1.8  million  of  local 
monies.  However,  San  Francisco  exceeds  these 
requirements  by  spending  local  resources  above  the 
matching  amount.  The  OOA  is  allocating  a large 
portion  of  its  local  discretionary  monies  to  the  same 
services  that  are  funded  through  Federal  and  State 
dollars  even  though  it  is  not  obliged  to  do  so.  While 
these  programs  are  varied  and  include  important 
services  such  as  congregate  meals  and  home-delivered 
meals,  they  may  not  be  the  services  most  needed  by 
•San  Franciscans.  The  OOA  could  re-direct 

approximately  $10  million  of  local  funds  to  meet  other  critical  needs  of  seniors  and  adults 
with  disabilities.  For  example,  at  a cost  of  approximately  $1  million,  the  OOA  could 
roughly  double  the  units  provided  for  some  of  the  services  that  address  housing, 
transportation  and  support  needs  such  as  grants-in-aid  for  emergency  housing  assistance 
($40,000  to  provide  an  additional  100  grants),  group  van  rides  ($680,000  for  an 
additional  40,000  trips)  and  personal  care,  homemaker,  and  chore  services  ($280,000  for 
an  additional  17,100  hours  of  service).  Any  funding  allocation  must  of  course  take  into 
account  the  impact  on  current  services  and  clients  and  the  limited  resources  available  to 
the  OOA. 

The  audit  recommends,  as  noted  above,  that  an  improved  needs  assessment  and 
allocation  process  identity  critical  needs  of  the  City’s  senior  and  disabled  community  and 
target  funds  to  those  needs.  The  Department  should  clearly  identity  State  and  Federal 
funding  amounts,  local  match  requirements,  and  any  discretionary  local  dollars 
committed  to  a particular  program.  The  Department  should  consider  directing  local 
resources  to  services  that  match  the  needs  most  frequently  identified  in  the  Department’s 
needs  assessment  activities.  Even  for  areas  of  need  such  as  housing  where  the  funding 


The  OOA  overmatches  its 
local  funding  for  Federal 
and  State  mandated 
programs  by  an  estimated 
$10  million.  These 
discretionary  local  dollars 
could  be  directed  to  other 
services  for  seniors  and 
adults  with  disabilities  or 
for  any  other  public 
purpose. 
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available  to  the  Department  may  not  be  enough  to  provide  the  primary  solution  to  the 
need,  the  Department  could  use  local  funds  for  planning  or  for  leveraging  funds  to 
address  the  problems  of  senior  and  disabled  communities  with  a coordinated  approach. 

FINDING  2-A.  CONTRACT  MONITORING  PROCEDURES 

Contracted  services  are  inconsistently  assessed  and 
monitored  by  the  Department.  Some  providers  are 
allowed  to  administer  their  own  contract  monitoring 
assessments,  while  others  have  them  administered  by 
Program  Analyst  staff.  Departmental  manuals  do  not 
include  basic  contract  monitoring  policies  and 
procedures,  follow-up  procedures,  or  documentation 
requirements.  Some  of  the  files  reviewed  by  the  audit 
lacked  any  evidence  of  follow-up  to  deficiencies  and 
others  showed  that  follow-up  varied  depending  on  the 
Program  Analyst  assigned.  Program  Analysts  have 
different  understandings  and  practices  regarding  which 
monitoring  tools  are  required  and  used  in  the  Department.  Minimal  managerial  oversight 
of  Program  Analysts,  lack  of  job-specific  training  for  contract  monitoring,  and  lack  of 
rotation  of  contract  monitoring  assignments  among  Program  Analysts  contribute  to  the 
inconsistency.  The  situation  undermines  the  relationship  between  contractors  and  the 
Department  and  can  result  in  the  Department  failing  to  identify  under-performing  and/or 
high-performing  contractors. 

The  audit  recommends  that  the  Department  develop  a policies  and  procedures  manual 
that  provides  information  regarding  all  contract  monitoring  processes,  the  types  of 
supporting  documentation  that  should  be  retained,  guidelines  for  the  use  of  contract 
monitoring  tools,  a schedule  of  the  expected  frequency  of  monitoring  procedures, 
identification  of  mandatory  procedures,  methodology  for  testing,  requirements  for  staff 
rotation,  review  procedures  for  management  staff  and  comprehensive  follow-up 
procedures  for  deficiencies.  Management  should  emphasize  consistent  monitoring  and 
should  employ  random  sampling  to  assess  whether  contracts  are  being  properly  and 
consistently  monitored.  The  Department  should  rotate  monitoring  assignments  among 
the  Program  Analysts  and  should  implement  a training  program  that  includes  specific 
contract  monitoring  training.  The  Department  should  fill  the  vacant  Program  Manager 
position  to  provide  the  Program  Analysts  with  sufficient  oversight,  guidance  and  training. 


Contract  monitoring  is 
inconsistent  and  weak. 
Some  providers  are  doing 
their  own  assessments. 
Monitoring  tools, 
documentation  and  follow- 
up are  not  consistent. 
Training  and  rotation  of 
Program  Analyst  staff  do 
not  occur  and  management 
oversight  is  minimal. 
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FINDING  2-B.  CONTRACT  MONITORING  INFORMATION 


Contractors  are  not  being  adequately  evaluated  on  the 
quality  of  services  because  existing  monitoring  tools 
focus  on  compliance  with  State  and  Federal 
requirements  and  on  unit  numbers  of  service. 
Monitoring  tools  are  general  in  nature  and  do  not 
capture  sufficient  program-specific  data  or  service 
issues  with  the  exception  of  nutrition  programs  that  are 
monitored  by  Nutritionists  under  State  regulations.  While  monitoring  tools  for  nutrition 
programs  include  qualitative  standards,  as  a whole,  the  Department’s  monitoring  tools  are 
deficient  and  the  three  main  contract  monitoring  tools  identified  by  staff  are  designed  to 
capture  quantitative  and  compliance  oriented  data  and  collect  little  information  about  the 
quality  of  the  services  being  provided.  The  lack  of  qualitative  and  program-specific  data 
makes  it  difficult  to  determine  whether  contractors  are  providing  seniors  or  adults  with 
disabilities  with  good  quality  service.  In  some  cases,  gaps  in  information  are  being  filled 
with  separate  monitoring  processes  and  tools.  Staff  effort  is  duplicative  and  some 
contractors  are  required  to  participate  repeatedly  in  segmented  monitoring  activities. 

The  audit  recommends  that  the  Department  implement  new  contract  monitoring  tools  or 
modify  existing  tools  in  order  to  measure  the  quality  of  services  rendered  and  provide 
specific  information  on  the  services  provided.  For  example,  monitoring  tools  should 
measure  the  quality  of  the  health  services,  the  effectiveness  of  the  translation  services, 
and  the  quality  of  the  counseling  services  being  provided  by  the  29  Community  Services 
contractors  and  not  just  the  number  of  hours  of  activity  scheduling,  number  of  hours  of 
translation,  and  number  of  social  services  hours  provided.  The  new  or  revised  contract 
monitoring  tools  should  also  include  measures  of  the  specific  services  provided  rather 
than  just  an  overall  rating  of  the  contractor. 

FINDING  2-C.  SF  GETCARE  DATA  SYSTEM 


The  Department  uses  an  electronic  database  system 
called  SF  GetCare  to  monitor  the  units  of  service 
provided  by  contractors.  Reports  from  the  system 
contain  inaccurate  information  and  are  unreliable  for 
purposes  of  measuring  disparities  between  the 
contracted  units  of  service  and  the  actual  units  of 
service  provided.  Department  staff  re-creates  the 
information  in  spreadsheets  because  the  SF  GetCare 
reports  are  not  reliable,  and  providers  have  reported  that  they  also  maintain  separate 
tracking  documents.  Incorrect  data  makes  it  difficult  to  determine  whether  contractors 
are  complying  with  the  terms  of  their  contracts  and  can  lead  to  the  Department  failing  to 
identify  contractors  that  are  not  providing  the  expected  units  of  service.  Contractors 
whose  performance  is  evaluated  based  on  SF  GetCare  reports  cannot  be  sure  their  service 
delivery  is  reflected  accurately.  The  City  has  spent  approximately  $2.6  million  on  SF 
GetCare,  including  grant  and  general  fund  monies  from  FY  2000-01  to  the  present.  The 
Human  Services  Agency  is  now  replacing  SF  GetCare  through  a combination  of  the 
Department’s  own  information  technology  staff  and  a new  contractor. 


SF  GetCare  reports  are 
unreliable  and  inaccurate, 
making  it  difficult  for  the 
Department  to  conduct 
accurate  assessments  of 
services  provided  by 
contractors. 


DAAS  contract  monitoring 
tools  emphasize  units  of 
service  and  compliance 
data  but  are  not  designed 
to  adequately  capture 
contractor  performance. 
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The  audit  recommends  that  the  Department  develop  a database  and  tracking  tools  that 
are  accurate,  reliable,  and  do  not  require  duplication  of  efforts  by  the  Program  Analysts. 
The  Department  should  establish  a regular  program  of  testing  the  accuracy  of  service  data 
and  reports.  As  of  the  writing  of  this  report,  the  Department  is  working  to  replace  SF 
GetCare  with  a combination  of  in-house  staff  and  a new  contractor.  The  City  should  also 
review  its  arrangements  with  the  current  contractor,  RTZ  Associates,  and  determine 
whether  service  expectations  and  obligations  have  been  met. 

FINDING  2-D.  HUMAN  SERVICES  AGENCY  MONITORING 


The  Department  of  Human  Services’  Office  of 
Contract  Management  (OCM)  is  assuming 
responsibility  during  FY  2004-05  for  DAAS  contract 
administration  and  monitoring.  In  general,  OCM  does 
conduct  fiscal  monitoring  but  does  not  retain  evidence 
of  monitoring  in  contract  files  and  also  lacks  adequate 
policies  and  procedures  for  administration  of  its 
contracts.  In  order  to  abate  the  weaknesses  identified 
in  the  audit,  OCM  needs  to  address  the  broader  need  for  improved  contract  monitoring  at 
DAAS  and  should  not  rely  solely  on  their  own  procedures. 

The  audit  recommends  that  OCM  develop  a policies  and  procedures  manual  for  contract 
monitoring  that  provides  detailed  information  regarding  how  fiscal  and  program 
monitoring  of  DHS  contracts  should  be  conducted,  including  monitoring  frequency,  the 
types  of  testing  that  should  be  performed  during  a monitoring  visit,  and  the  types  of 
supporting  documentation  that  should  be  retained.  To  the  extent  that  existing  policies 
and  procedures  used  by  DAAS  are  applicable,  they  should  be  incorporated  into  the  new 
manual.  OCM  should  also  develop  a site  visit  summary  form  (or  modify  the  existing  site 
visit  form)  to  provide  information  on  fiscal  and  programmatic  monitoring  activities, 
results,  and  corrective  actions  taken  and  to  serve  as  a basis  for  the  report  that 
accompanies  contract  renewal  requests  to  the  Human  Services  Commission. 

FINDING  3-A.  RFP  PROCESS  AND  SELECTING  CONTRACTORS 

The  Department  did  not  adhere  to  the  rules  and 
guidelines  it  established  to  govern  its  own  granting 
process.  A Request  for  Proposals  (RFP)  was  issued  in 
2003  for  three-year  contracts  beginning  in  FY  2003-04 
with  the  primary  criteria  that  bidders  scoring  under  70 
points  would  not  be  considered.  However,  16  bids 
that  scored  under  70  points  were  funded  and  a variety 
of  other  decisions  were  not  reflective  of  the  scores  and 
rankings.  The  Department  funded  contractors  who 
had  had  a prior  contract  even  if  that  meant  not  funding  bidders  that  scored  and  ranked 
higher  as  a result  of  the  RFP  process.  The  Department  received  $1.16  million  of  funding 
above  its  original  budget  during  the  RFP  process  but  distributed  those  funds  without 


Consistent  and  objective 
criteria  were  not  used  to 
select  all  contractors  and  to 
set  contractor  funding 
amounts  and  resulted  in 
DAAS  awarding  a contract 
to  a poorly  qualified  service 
provider. 


The  Department  of  Human 
Services’  Office  of  Contract 
Management  contract 
monitoring  procedures  will 
not  abate  the  issues 
identified  with  DAAS 
program  monitoring. 
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regard  to  RFP  scores  and  rankings.  The  Department  gave  incomplete  and  inconsistent 
justifications  for  funding  decisions  that  deviated  from  the  RFP  score  and  rankings.  For 
example.  West  Bay  Pilipino  was  not  funded  for  a Community  Services  contract  despite 
the  fact  that  they  scored  higher  than  five  other  bidders,  four  of  whom  scored  under  70 
points.  Chinese  Newcomer  scored  ten  points  higher  than  La  Raza  Centro  Legal  but  was 
not  funded  for  a Naturalization  Services  contract,  with  the  Commission  simply  stating 
that  La  Raza  was  an  important  part  of  the  Naturalization  network.  These  decisions 
resulted  in  discontent  among  bidders  and  the  appearance  of  unfairness  in  the  RFP 
process.  The  Department  may  also  have  contracted  for  services  from  providers  that  are 
not  the  most  qualified.  In  at  least  one  case  a contract  was  awarded  to  a provider,  Filipino 
American  Council,  that  failed  to  perform,  did  not  respond  to  departmental  efforts  to  gain 
compliance,  and  was  later  de-funded. 

The  audit  recommends  that  the  Department  follow  the  rules  it  establishes  for  the  RFP 
process.  Competitive  funding  guidelines  and  rules  for  selecting  contractors  should  be 
established  and  communicated  to  bidders  in  the  RFP  packet  and  should  not  be  changed 
during  the  process.  The  Department  must  clearly  state  in  the  packet  how  it  plans  to  use 
RFP  scores  and  rankings.  Any  funding  decisions  that  differ  from  the  rules  and  guidelines 
should  be  documented  and  the  reasoning  behind  them  consistently  applied.  The 
Department  may  have  reasons  for  deviating  from  its  rules  or  guidelines,  however  the 
justification  and  logic  for  funding  particular  contractors  and  for  granting  particular 
amounts  should  be  consistent. 

FINDING  3-B.  LOBBYING  EFFORTS 

The  approaches  used  by  the  Department  and 
Commission,  and  the  levels  of  review  in  the  RFP 
process  create  opportunities  for  changes  that  may  not 
be  consistent  with  the  intent  of  a fair  and  competitive 
allocation  process.  A situation  was  created  that  pitted 
contractors  against  each  other  and  encouraged 
lobbying  efforts  to  receive  more  funding.  Both  the 
Department  and  the  Commission  appeared  to  have 
been  influenced  by  advocacy  efforts  and  special 
considerations  were  given  to  certain  bidders  that  were 
not  analyzed  for  accuracy  or  applied  equally  to  all 
bidders  within  the  service  categories.  If  the  Department  or  the  Commission  make  special 
considerations  and  do  not  follow  their  RFP  rules  and  guidelines,  these  considerations 
should  be  thoroughly  analyzed  and  applied  equally.  Although  input  from  the  public  is  a 
valuable  component  of  any  RFP  process,  this  input  should  be  considered  in  support  of, 
and  not  in  conflict  with,  the  RFP  scores. 

The  audit  recommends  that  the  Department  streamline  and  define  its  RFP  process  so 
that  the  feedback  received  during  the  RFP  process  can  be  considered  in  light  of  the  RFP 
results  and  not  in  lieu  of  the  scores  and  rankings.  Feedback  received  should  be  compared 
to  each  bidder’s  RFP  evaluation,  the  Department’s  needs  assessment,  and  the  priorities  of 
the  Department.  This  comparison  should  be  consistent  from  contractor  to  contractor 


Lobbying  efforts  influenced 
the  Department  and  the 
Commission’s  funding 
decisions  and  the 
justifications  used  for 
selecting  certain 
contractors  were  sometimes 
inconsistent  and/or 
unsupported  by  analysis. 
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within  the  specific  service  categories.  The  Department  should  adhere  to  its  funding 
recommendations  unless  substantial  evidence  is  presented  that  demonstrates  the  need  for 
making  a change  or  an  exception  to  the  funding  rules  and  guidelines.  In  doing  so,  the 
Department  must  ensure  that  the  same  considerations  are  given  to  every  bidder  within  the 
service  category. 

FINDING  3-C.  FUNDING  ALLOCATIONS  OUTSIDE  THE  RFP  PROCESS 

The  Department  faces  uncertainty  regarding  its  revenue 
streams  and  typically  receives  different  amounts  of 
funding  than  it  originally  anticipates.  The  manner  in 
which  the  Department  funnels  changes  in  funding  to 
contractors  is  subject  to  the  same  weaknesses  identified 
in  the  RFP  process.  The  Department  lacks  a plan  or 
policy  that  states  how  changes  in  funding  should  be 
allocated.  The  Department  has  made  funding  changes  to  specific  programs  and 
contractors  without  citing  the  needs  of  the  customers  impacted  by  those  changes  or 
stating  what  guidelines  they  were  following  for  their  decisions. 

The  audit  recommends  that  the  Department  create  a policy  document  or  funding 
guidelines  that  reflect  how  changes  will  be  funneled  to  contractors  if  revenue  streams 
change.  The  plan  must  prioritize  services  and  needs  in  a logical  and  competitive 
manner — providing  more  or  fewer  units  for  existing  services,  moving  to  a ranked  list  of 
services  that  could  not  be  funded  with  the  original  budget,  or  other  approaches  supported 
by  analysis.  Changes  to  Federal  and  State  revenues  that  result  in  changes  to  allocations 
for  particular  contractors  must  be  clearly  accounted  for.  The  Department  appears  to  have 
made  some  strides  in  this  area,  but  it  needs  to  continue  to  improve  the  transparency  of 
changes  to  contract  funding  resulting  from  changes  in  revenues. 

FINDING  4-A.  INFORMATION,  REFERRAL,  AND  ASSISTANCE  SERVICES 

The  Department  funds  Information,  Referral,  and 
Assistance  (I,  R & A)  services  three  ways;  through  the 
ten  Neighborhood  Resource  Centers  (NRCs),  the 
Department’s  internal  I,  R & A unit  and  as  part  of  the 
services  that  some  contractors  must  provide  under 
other  contracts,  spending  over  SI. 6 million  in  this 
area.  The  Department  lacks  consistent  definitions 
among  the  three  different  I,  R & A providers  for  what  should  count  as  a unit  of 
information,  referral,  follow-up,  or  assistance.  Contractors  who  provide  these  services  do 
not  track  them  independently  and  referral  procedures  between  I,  R & A providers  may 
lead  to  duplicate  counts.  The  Department  cannot  adequately  assess  the  degree  of  overlap 
or  maximize  the  efficiency  of  the  I,  R & A services  it  provides. 

The  audit  recommends  that  the  Department  clearly  define  I,  R & A services  and 
consistently  utilize  one  set  of  definitions  for  all  DAAS  funded  I,  R & A service  providers. 
The  Department  should  modify  data  collection  procedures  in  order  to  gather  I,  R & A 
service  data  for  DAAS  funded  contractors  in  enough  detail  to  provide  for  analysis  and 


The  Department  does  not 
have  adequate  information 
about  the  results  and 
outcomes  of  its 
information,  referral  and 
assistance  services. 


The  manner  in  which  the 
Department  funnels 
changes  in  funding  to 
contractors  is  subject  to 
the  same  weaknesses  as 
the  RFP  process. 
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comparisons  among  the  I,  R & A service  providers.  The  Department’s  internal  I,  R & A 
unit  and  the  NRCs  should  track  and  aggregate  data  regarding  the  type  of  information 
supplied  to  consumers,  the  outcome  of  follow-up  activities,  and  the  source  of  all 
incoming  referrals.  Simple  referrals  made  among  the  providers  should  not  be  counted 
twice.  Service  numbers  for  the  three  separate  I,  R & A service  providers  should  be 
combined  and  analyzed  to  determine  the  scope  of  the  Department’s  I,  R & A efforts. 
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INTRODUCTION 


In  fulfilling  the  mission  of  the  Office  of  the  Controller  to  promote  efficient,  effective, 
and  accountable  government  within  the  City  and  County  of  San  Francisco  (City),  the 
Controller’s  City  Services  Auditor  conducted  a performance  audit  of  the  Department 
of  Aging  and  Adult  Services  (DAAS).  The  Department  of  Aging  and  Adult  Services 
includes  the  Public  Guardian,  Public  Administrator,  Public  Conservator,  Office  on  Aging 
(and  its  Commission),  the  County  Veterans  Service  Office,  and  the  Adult  Protective 
Services  unit.  In  July  2004,  the  Department  of  Aging  and  Adult  Services  and  the 
Department  of  Human  Services  (DHS)  were  placed  under  the  umbrella  of  a single 
Human  Services  Agency  (HSA).  The  purpose  was  to  benefit  from  economies  of  scale 
that  come  from  integrating  fiscal,  planning,  contracting,  administrative  reporting  and 
audit  responsibilities. 

AUDIT  SCOPE  & OBJECTIVES 

The  current  performance  audit  of  the  Department  of  Aging  and  Adult  Services  covers 
only  the  services  provided  by  the  Office  on  Aging  (00  A).  Other  units  of  the 
Department,  including  the  Public  Administrator/Public  Guardian  and  Public  Conservator, 
have  been  separately  audited  in  recent  years  and  were  not  included  in  this  analysis.  The 
OOA  acts  primarily  as  a contracting  agency  with  community-based  non-profit  groups  to 
provide  direct  services  to  seniors  and  adults  with  disabilities.  The  OOA  funds  services 
worth  an  estimated  $17  million  annually,  ranging  from  home-delivered  meals  to  legal 
services.  A listing  of  all  contracted  services  by  program  area  and  their  respective  dollar 
amounts  for  FY  2004-05  is  included  in  Appendix  A. 

The  performance  audit  objectives  were  to  evaluate  how  well  the  Department  determines 
the  needs  of  seniors  and  adults  with  disabilities,  how  effectively  the  Department  monitors 
contractor  performance,  how  well  the  Request  for  Proposal  (RFP)  process  used  to  select 
contractors  worked,  and  the  efficiency  of  the  information,  referral  and  assistance  services 
the  Department  provides.  Since  the  OOA  provides  little  direct  service  and  is  essentially 
funding  contractors  with  a mix  of  Federal,  State  and  local  funds  for  seniors  and  adults 
with  disabilities,  the  Department  should  develop  a strong  capacity  to  determine  the  needs 
of  seniors  and  adults  with  disabilities,  contract  for  services  that  meet  those  needs,  and 
monitor  contractor  performance.  The  merger  of  DAAS  and  DHS  into  one  agency  creates 
an  opportunity  to  examine  these  OOA  processes  and,  where  improvement  is  needed, 
implement  the  recommendations  of  this  audit  report  into  the  changing  organizational 
structure. 

METHODOLOGY 

The  performance  audit  involved  a review  of  DAAS’  contracts  information,  year-end 
reports,  the  Department’s  State-mandated  Area  Agency  Plan  and  its  yearly  updates,  needs 
assessment  studies,  contract  monitoring  reports,  Commission  on  Aging  meeting  minutes, 
California  Department  on  Aging  Memos  and  Regulations,  Federal  and  State  statutes, 
benchmarking  information  from  other  counties,  and  other  pertinent  documentation  and 
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historical  records.  Audit  staff  also  conducted  multiple  interviews  with  DAAS  and  DHS 
staff,  the  Commission  on  Aging  President,  DAAS  contractors,  and  stakeholders.  The 
audit  also  included  a focus  group  with  approximately  35  stakeholders  and  contractors  to 
solicit  feedback  regarding  the  Department.  Site  visits  and  meetings  with  the  staff  were 
conducted  at  the  ten  Neighborhood  Resource  Centers  for  Seniors  and  Adults  with 
Disabilities.  Much  of  the  information  collected  from  the  focus  group  and  site  visits 
reinforced  some  of  the  audit  findings  and  the  information  helped  inform  the  substance  of 
this  report.  We  conducted  this  audit  according  to  standards  for  the  professional  practice 
of  internal  auditing.  We  limited  our  review  to  those  areas  specified  in  the  audit  scope 
section  of  this  report. 
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CHAPTER  1 

THE  DEPARTMENT  OF  AGING  AND  ADULT  SERVICES  DOES 
NOT  ADEQUATELY  IDENTIFY  NEEDS,  ESTABLISH  SERVICE 
PRIORITIES  OR  JUSTIFY  THE  LEVEL  OF  FUNDING 
ALLOCATED  TO  ITS  CONTRACTED  SERVICES 


CHAPTER  SUMMARY 

The  needs  assessment  process  conducted  by  the  Department  of  Aging  and  Adult 

Services  (DAAS)  does  not  adequately  identify  the  needs  of  San  Francisco’s  older 
and  disabled  adult  populations  or  analyze  how  needs  are  addressed  by  the  services 
it  funds.  The  needs  assessment  does  not  adequately  prioritize  among  or  establish  goals 
and  objectives  for  the  populations  served.  As  a result,  the  City  cannot  be  confident  that  it 
is  serving  the  most  critical  needs  of  San  Francisco’s  seniors  and  adults  with  disabilities 
and  cannot  assure  that  it  is  using  its  resources  effectively.  To  correct  this,  the 
Department  should  use  the  information  from  its  needs  assessment  activities  to  identify 
specific  needs  and  target  populations,  identify  existing  resources  and  gaps  in  service, 
establish  service  priorities  based  on  needs  identified  and  gaps  in  service,  set  goals  and 
objectives  on  the  basis  of  the  priorities,  and  allocate  funds  appropriately. 


THE  DEPARTMENT  CONDUCTS  STATE-MANDATED  NEEDS  ASSESSMENT 
ACTIVITIES.  HOWEVER  THE  ANALYSIS  DOES  NOT  IDENTIFY  SPECIFIC 
NEEDS  AND  TARGET  POPULATIONS,  IDENTIFY  EXISTING  RESOURCES 
AND  GAPS  IN  SERVICE,  ESTABLISH  PRIORITIES,  OR  ALLOW  THE 
DEPARTMENT  TO  ALLOCATE  RESOURCES  EFFECTIVELY. 


San  Francisco  researches  the  needs  of  the  City’s  senior  and  disabled  adult  populations  as 
part  of  its  compliance  with  Federal  and  State  funding  and  regulatory  requirements.  Every 
four  years,  the  City  must  develop  a State-mandated  Area  Plan.  State  regulations  specify 
that  the  Area  Plan  should  include  a systematic  process  for  collecting,  analyzing,  and 
interpreting  timely  information  on  local  service  needs  of  the  client  population  including 
both  seniors  and  functionally  impaired  adults.  This  needs  assessment  should  identify 
existing,  unmet  and  potential  needs,  identify  target  populations,  determine  the  services 
and  resources  that  exist,  identify  any  underutilized  services,  and  identify  any  barriers  to 
access.  The  process  should  also  research  and  determine  the  full  extent  of  needs,  even  if 
the  State  does  not  provide  funding  for  some  of  the  areas  identified.  The  information 
collected  should  allow  for  a “gap”  analysis  of  the  difference  between  the  critical  needs  of 
the  senior  population  and  what  is  available  through  government  and  other  service 
agencies.  The  Area  Plan  regulations  further  specify  that  counties  should  identify 
priorities,  set  goals  and  objectives,  and  focus  on  a specific  range  or  group  of  needs  and 
target  populations. 
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The  table  below  summarizes  the  California  Department  of  Aging  (CDA)  requirements, 
what  the  Department  does  to  comply,  and  the  effects  of  this  information.  Specific 
weaknesses  in  these  processes  are  then  discussed  in  more  detail  in  Table  1. 

Table  1 


What  the  CDA  requires 

What  DAAS  Does 

Result/Effect 

Needs  Assessment  Activities 

The  needs  assessment  should  elicit 
data  about  the  client  population, 
using  surveys  that  address  the  State’s 
core  questions  including  age,  race, 
ethnicity  and  other  demographics. 

DAAS  conducts  surveys  and  other 
activities  that  meet  the  State’s 
minimum  requirements  and  does 
additional  needs  assessment  activities 
such  as  phone  surveys,  focus  groups, 
case  file  assessments,  and  collection 
of  statistics  and  reports  done  by 
various  agencies. 

DAAS  compiles  several 
sources  of  information 
about  seniors  and  adults 
with  disabilities. 

Identify  Specific  Needs 

The  Area  Plan  should  identify  the 
types  and  extent  of  existing  and 
potential  needs  of  the  client 
population  within  the  community. 

DAAS  identifies  several  needs,  but 
the  Area  Plan  does  not  provide 
adequate  information  on  the  extent  of 
those  needs,  the  specific  populations 
with  those  needs  or  where  those 
populations  are  located. 

Do  not  know  how  much 
need  for  particular 
services  exists  or  the  type 
and  extent  of  needs. 

Identify  Target  Populations 

The  Area  Plan  should  identify 
specific  target  populations,  targeting 
priorities,  any  barriers  to  services 
target  populations  may  have,  and 
methods  to  be  used  to  address  their 
needs. 

The  Area  Plan  states  that  it  targets 
particular  populations,  but  there  is  no 
use  of  demographic  information  to 
indicate  how  it  identifies  target 
populations  and  services,  sets 
priorities,  or  chooses  methods  used  to 
address  the  needs  of  target 
populations. 

Do  not  know  how 
populations  are  being 
targeted  for  services. 
Have  to  rely  on  the 
Department’s  statement 
that  specific  populations 
are  targeted  in  the 
services  funded. 

Identify  Existing  Services,  Resources 
and  Constraints 

The  Area  Plan  should  identify  the 
services  and  resources  existing 
within  the  City  that  are  available  for 
addressing  identified  needs  as  well  as 
existing  constraints. 

The  Area  Plan  lists  other  local 
resources,  including  services 
provided  by  other  City  departments 
and  by  other  institutional  and 
community-based  service  providers, 
but  the  Area  Plan  does  not  discuss 
constraints  or  how  these  other  service 
providers  complement  services 
provided  by  the  Department. 

DAAS  might  be 
providing  duplicate 
services  and  might  be 
allocating  its  funds  to 
services  that  are  already 
being  provided  elsewhere. 

Identify  Gaps  in  Service 

The  Area  Plan  should  identify  gaps 
in  services  by  comparing  existing 
services  to  the  range  of  unmet  needs, 
under-utilized  services,  and  barriers 
that  prevent  access.  This  “gap 
analysis”  should  help  suggest 
priorities  for  programs  and  services 
and  where  they  should  be  targeted. 

The  Area  Plan  does  not  show  an 
analysis  of  gaps  in  service  compared 
to  existing  services  and  resources. 

Without  knowing  the  gap 
between  what  currently 
exists  and  what  needs  are 
unfulfilled,  DAAS  cannot 
effectively  identify  how 
programs  and  services 
should  be  prioritized  or 
allocate  funds  to  close  the 
gaps. 
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What  the  CPA  requires 

Establish  Priorities 


What  DAAS  Does 


Result  Effect 


The  Area  Plan  should  contain  the 
identification  in  priority  order  of  the 
areas  of  need  resulting  from  the 
assessment  process.  The  needs 
identified  by  the  needs  assessment 
should  lead  directly  to  the 
establishment  of  priorities. 


The  Area  Plan  lists  the  findings  of  The  overall  needs 
each  of  the  needs  assessment  assessment  findings  are  so 

activities.  However,  the  Area  Plan  broad  that  they  cannot  be 
does  not  show  an  analysis  or  method  used  as  a tool  for 
for  prioritizing  those  findings  and  prioritizing  sen  ices  the 
simply  states  that  overall  needs  are  Department  should  fund, 
housing,  transportation  and  in-home 
sendees. 


Set  Goals  & Objectives 


The  Area  Plan  lists  three  goals  and 
the  objectives  for  each  goal. 
However,  the  goals  primarily  address 
improvements  to  the  sendee  delivery 
system  rather  than  focusing  efforts 
on  needs  identified. 


Without  goals  and 
objectives  based  on 
priorities,  the  Area  Plan 
may  not  provide  the 
Department  with 
sufficient  direction  or 
guidance  for  providing 
sendees  to  those  most  in 
need. 


On  the  basis  of  identified  priorities, 
goals  and  objectives  should  be 
developed  to  guide  the  planning 
efforts  to  focus  on  a specific  range  or 
group  of  needs. 


The  Area  Plan  Does  Not  Identify  Needs  of  Specific 
Segments  of  the  Population  or  How  Funded 
Services  Meet  the  Needs  of  Targeted  Populations 

Under  State  guidelines,  the  Area  Plan  should  identify  the  types  and  extent  of  existing  and 
potential  needs  of  the  client  population  within  the  community.  The  Department's  needs 
assessment  identifies  very  broad  categories  of  need  but  does  not  provide  adequate 
information  on  the  specific  populations  in  need  such  as  their  locations,  the  extent  or  type 
of  need,  or  the  methods  to  meet  them. 

For  example,  Federal  mandates  of  the  Older  Americans  Act  state  that  services  must  pay 
particular  attention  to  low-income  seniors.  To  identify  low-income  populations  of 
seniors,  the  Area  Plan  should  include  information  on  the  income  levels  of  seniors  in  San 
Francisco  and  where  the  population  is  concentrated,  but  the  Area  Plan  does  not  include 
any  demographic  information  that  would  accomplish  this.  The  regulations  also 
emphasize  targeting  funds  and  services  to  non  or  limited-English  speaking  persons, 
among  other  groups.  However,  the  Area  Plan  does  not  discuss  the  extent  or  type  of  needs 
among  this  group  or  the  services  available  to  them,  such  as  the  availability  of  translation, 
naturalization  and  legal  services.  Similarly,  the  regulations  emphasize  targeting  funds 
and  services  to  frail  older  persons.  However,  the  Area  Plan  does  not  discuss  which 
services  are  available  to  this  group  or  the  extensive  set  of  needs  of  this  population. 

In  other  area  plans  reviewed,  counties  provide  specific  descriptions  of  how  the  needs  of 
the  target  populations  will  be  addressed.  For  example,  the  Silicon  Valiev  Area  Plan 
targets  specific  populations  of  individuals  living  in  geographic  areas  of  the  county  that 
are  in  most  need.  These  populations  include  persons  living  in  districts  of  San  Jose  or 
other  parts  of  Santa  Clara  County  that  have  incomes  below  S25,000  or  who  have 
difficulties  going  outside  the  home,  caring  for  personal  needs,  have  transportation 
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difficulties,  or  live  alone.  Six  specific  target  populations  and  the  barriers  each  experience 
are  identified.  The  Silicon  Valiev  Area  Plan  shows  persons  in  greatest  economic  need 
are  in  Downtown  and  East  Valley  districts  and  in  South  Valley  of  the  county  and  includes 
estimates  of  the  numbers  of  people  with  incomes  below  $25,000.  Persons  of  greatest 
economic  need  require  services  that  supplement  income  like  food  and  nutrition  programs, 
legal  services  to  assist  with  problems  with  benefits,  information  on  subsidized  housing, 
and  assistance  with  job  training  and  placement.  Individuals  with  greatest  social  need  are 
shown  to  live  in  the  North  county  and  West  Valley  areas.  Persons  with  greatest  social 
need  require  case  management,  personal  care  and  home  care  services,  home  delivered 
meals  and  respite  care. 

The  Area  Plan  Does  Not  Adequately 
Identify  Existing  Resources  or  Constraints 
or  Perform  a Gap  Analysis 

The  Area  Plan  should  identify  the  services  and  resources  existing  within  the  City  that  are 
available  for  addressing  identified  needs  as  well  as  existing  constraints.  San  Francisco’s 
Area  Plan  lists  other  local  resources,  including  services  provided  by  other  City 
departments  and  by  other  institutional  and  community-based  service  providers.  For 
example,  the  Area  Plan  states  that  there  are  111  senior  centers  and  clubs  that  provide 
social/recreational/educational  programs  throughout  the  City.  Besides  these  1 1 1 centers 
that  provide  these  services,  the  Department  contracts  with  29  contractors  to  provide 
community  services  and  also  funds  nutrition  education,  community  education,  housing 
education,  and  medication  education  through  several  other  types  of  contracts.  However 
the  Area  Plan  does  not  discuss  the  scope  of  services  provided  by  the  1 1 1 centers  or  how 
the  services  the  Department  contracts  out  complement  or  overlap  with  the  services 
provided  by  these  other  1 1 1 centers. 

The  Area  Plan’s  weakness  in  analyzing  complementary  services  and  gaps  is  particularly 
problematic  in  its  application  to  housing.  San  Francisco’s  housing  affordability  problems 
are  well  documented,  and  the  needs  assessment  studies  conducted  by  the  Department 
specifically  identify  housing  as  the  most  prevalent  need  for  seniors  and  adults  with 
disabilities.  DAAS  itself  does  not  receive  or  manage  significant  funding  for  housing, 
however,  other  City  agencies  do  fund  housing  services  for  San  Francisco's  seniors  and 
adults  with  disabilities.  The  Area  Plan  does  list  four  major  programs  that  the  Mayor's 
Office  offers  for  low-income  seniors  including  an  internet-based  housing  resource 
database,  funds  for  nonprofit  housing  development  corporations  to  develop  senior 
housing  projects,  two  programs  which  fund  loans  and  grants  for  housing  code  upgrades, 
and  housing  enforcement  and  rehabilitation  for  low-income  seniors  and  adults  with 
disabilities.  However,  missing  from  the  Area  Plan  is  any  information  about  specific 
senior  populations  in  most  need,  a strategy  for  funding  housing  services  that  complement 
other  efforts  of  City  departments,  or  a projection  of  the  future  need  for  senior  housing  and 
strategies  that  might  address  it. 

Constraints  affecting  service  delivery  should  also  be  discussed  in  the  Department’s  Area 
Plan — such  as  an  outline  of  the  relevant  language  or  cultural  barriers,  access  to 
transportation,  or  affordability.  However,  the  only  discussion  of  constraints  is  that 
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inadequate  funding  is  available  to  provide  all  services.  Other  area  plans  we  reviewed  had 
specific  information  on  barriers  and  constraints.  For  example,  the  Alameda  County  Area 
Plan  discusses  the  barriers  that  prevent  older  adults  from  receiving  nutritional  needs, 
medical  assistance,  and  mental  health  assistance.  The  Alameda  County  Area  Plan  also 
talks  about  barriers  to  meeting  religious/spiritual  needs  and  identifies  transportation  as  a 
barrier  to  socialization.  The  Silicon  Valiev  Area  Plan  specifically  discusses  constraints  to 
providing  two  critical  needs  in  the  County — in-home  services  and  respite  care  to  prevent 
unnecessary  or  premature  institutionalization.  The  Silicon  Valiev  Area  Plan  discusses  the 
cost  of  these  services  and  the  limits  under  Medicare  and  most  private  health  insurance 
policies  that  constrain  the  county’s  ability  to  provide  them. 

The  Department’s  Area  Plan  should  identify  gaps  in  services  by  comparing  existing 
services  to  the  range  of  unmet  needs,  under-utilized  services,  and  barriers  that  prevent 
access.  This  “gap  analysis”  should  help  suggest  priorities  for  programs  and  services  and 
where  they  should  be  targeted.  The  gap  analysis  should  therefore  describe  the  following 
three  conditions: 

■ What-  What  are  the  unmet  needs  and  what  services  address  those  needs? 

■ How  large-  How  large  is  the  unmet  need? 

■ Where - Where  are  the  unmet  needs?  (Specific  geographical  area) 

San  Francisco’s  Area  Plan  does  not  show  any  analysis  of  gaps  in  service  compared  to 
existing  need,  services  and  resources.  Again,  other  area  plans  we  reviewed  included  a 
gap  analysis.  For  example,  the  Silicon  Valiev  Area  Plan  consolidates  the  results  of  the 
needs  assessment  data  collected,  including  specific  geographic  information,  and  states 
that  major  gaps  in  service  were  found  to  exist  with  the  availability  of  respite  programs  for 
caregivers,  for  individuals  who  experienced  difficulty  going  outside  of  home  and  caring 
for  personal  needs  with  no  one  available  to  provide  assistance. 

Without  adequate  knowledge  of  existing  services,  the  scope  of  resources  available,  and 
the  resulting  gaps  in  services,  DAAS  might  be  failing  to  provide  services  that  are 
critically  needed,  and/or  providing  duplicate  services. 

The  Area  Plan  Does  Not  Establish 
Adequate  Service  Priorities  Based  on 
Needs  Identified  in  the  Needs  Assessment 

The  Area  Plan  should  identify,  in  priority  order,  the  areas  of  need  resulting  from  the 
needs  assessment  process.  San  Francisco’s  Area  Plan  lists  the  findings  of  its  needs 
assessment  activities  but  does  not  provide  analysis  or  methods  for  prioritizing  those 
findings  and  simply  states  that  the  overall  needs  are  housing,  transportation  and  home  and 
community  based  services.  The  needs  assessment  findings  are  so  broad  that  they  cannot 
support  prioritization  of  the  services  the  Department  should  fund. 

Needs  assessment  activities  conducted  by  DAAS  such  as  focus  groups,  town  hall 
meetings,  and  other  reports  showed  a variety  of  needs  of  seniors  and  adults  with 
disabilities,  but  these  listed,  rather  than  prioritized,  needs  without  respect  to  severity  or 
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criticality.  The  Area  Plan  simply  states  in  a short  paragraph  that  as  a result  of  the  needs 
assessment,  the  findings  of  highest  priority  are  housing,  transportation,  and  home  and 
community-based  services  such  as  in-home  care,  case  management,  safety,  nutrition,  and 
money  management  programs.  These  broad  categories  of  need  do  not  allow  the  agency 
to  reach  conclusions  as  to  what  to  fund.  For  example,  there  are  many  types  of  housing 
needs  such  as  supportive  housing,  subsidized  housing,  and  housing  for  the  disabled,  some 
of  which  may  be  more  needed  than  others.  Frail  seniors  need  more  in-home  services 
while  non-English  speaking  seniors  need  more  translation  and  information  services. 

These  types  of  distinctions  and  service  priorities  are  absent  in  the  Area  Plan. 

Other  area  plans  we  reviewed  established  priorities,  addressed  the  needs  of  specific 
populations  based  on  targeting  requirements  under  Federal  and  State  law  and 
demonstrated  how  resources  would  be  allocated.  For  example,  Alameda  County 
identifies  all  services  being  provided  through  the  county  and  uses  an  allocation  formula 
county  wide  that  targets  funds  to  populations  of  older  adults  most  in  need  within  four 
geographic  areas.  In  a similar  fashion,  Santa  Clara  County  analyzed  its  needs  assessment 
findings  and  developed  two  key  prioritization  rules;  severity  of  needs  and  the  size  and 
characteristics  of  the  populations  affected.  When  taken  together,  the  information  and 
decision  rules  led  the  county  to  target  specific  populations  and  to  fund  16  priority 
services. 

Lack  of  Effective  Prioritization  of  Needs 
Prevents  DAAS  From  Developing 
Goals  and  Objectives  Based  on  Needs 

On  the  basis  of  identified  priorities,  goals  and  objectives  should  be  developed  to  guide 
the  planning  efforts  to  focus  on  a specific  range  or  group  of  needs.  The  Department’s 
Area  Plan  lists  three  goals: 

1)  Provide  an  integrated,  consumer-driven  system  to  meet  senior  needs  that 
reflects  the  diversity  of  San  Francisco  seniors; 

2)  Provide  ongoing  planning  efforts  to  ensure  appropriate  services  are  available 
to  San  Francisco  seniors  as  the  demand  for  home  and  community-base  services 
increases,  and; 

3)  Identify  and  develop  specific  targeting  methods  to  reduce  access  barriers  to 
isolated  older  and  disabled  adults. 

Rather  than  addressing  how  the  needs  identified  will  be  met,  these  goals  primarily 
address  improvements  to  the  service  delivery  system.  Two  of  the  goals  are  followed  by 
objectives  that  identify  units  of  service  and  populations  to  be  served,  but  it  is  not  clear 
that  these  derive  from  the  needs  identified  by  the  Area  Plan  research.  Without  a clear  link 
to  the  needs  assessment,  these  goals  and  objectives  do  not  provide  the  Department  or 
decision-makers  with  sufficient  direction  or  guidance  for  providing  service  or  making 
allocation  decisions. 

In  other  county  area  plans  we  reviewed,  goals  and  objectives  are  linked  to  priority  areas 
of  need  identified  in  the  needs  assessment.  For  example,  the  Silicon  Valley  Area  Plan 
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describes  goals  and  objectives  that  are  specific  to  the  services  to  be  provided  and  distinct 
from  other  goals  and  objectives  directed  toward  improving  or  enhancing  services, 
improving  the  sendee  delivery  system,  and  enhancing  funding  for  long-term  care 
services. 

In  summary,  San  Francisco’s  needs  assessment  process  does  not  effectively  accomplish 
the  purpose  of  determining  the  needs  seniors  and  adults  with  disabilities.  The  Area  Plan 
collects  information  on  the  population,  local  organizations  and  other  public  agencies  that 
deliver  sendees  to  the  older  and  disabled  adult  populations,  but  it  does  not  effectively 
compile  or  analyze  this  information.  Without  an  accurate  picture  of  needs  in  the 
community  and  the  services  being  provided,  it  is  not  possible  to  determine  where  gaps 
exist.  Consequently,  it  is  not  possible  to  determine  how  the  Department  should  prioritize 
or  allocate  resources  to  meet  the  needs  of  the  older  and  disabled  adult  community  in  San 
Francisco.  The  fact  that  San  Francisco  has  limited  resources  for  aging  and  adult  sendees 
makes  it  critical  to  correct  these  weaknesses  in  the  planning  process. 


FINDING  1-B.  THE  OFFICE  ON  AGING  S FUNDING  DISTRIBUTION  TO 
SPECIFIC  SERVICES  DOES  NOT  REFUECT  THE  NEEDS  MOST  OFTEN 
IDENTIFIED  BY  THE  DEPARTMENT’S  OWN  NEEDS  ASSESSMENT 
ACTIVITIES. 


There  are  a total  of  29  categories  of  sendees  currently  funded  by  the  Office  on  Aging 
(OOA)  of  which  the  top  five  funded  are  Congregate  Meals  (25%),  Home-Delivered 
meals  (21%),  Community  Sendees  (15%),  Resource  Centers  for  Seniors  and  Adults  with 
Disabilities  (7%),  and  Care  Management  (5%).  These  five  sendees  make  up  an  estimated 
73%  (S12.4  million)  of  the  total  OOA  contracted  sendees.  The  majority  of  spending  is 
for  Congregate  Meals  and  Home-Delivered  Meals,  which  make  up  around  46%  of  the 
funding.  The  OOA  spends  almost  $10  on  meals1  for  every  dollar  spent  on  transportation 
and  housing  combined.  The  total  amount  spent  on  meal  programs  is  also  more  than  the 
aggregate  amount  spent  on  every  other  program  (excluding  Community  Sendees).  A 
review  of  the  funding  allocation  also  shows  that  21  of  the  29  sendees  funded  by  the  OOA 
are  funded  for  less  than  2%  of  the  total  funding.  Appendix  A shows  the  amounts  and 
percentage  of  funding  each  service  receives  and  the  chart  below  illustrates  that  funding 
allocation. 


1 Meals  refer  to  the  Congregate  Meals,  Home  Delivered  Meals,  and  RDM  Clearinghouse  sen  ice 
categories. 
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Figure 
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The  OOA  receives  around  one-third  of  its  funding  for  contracted  services  from  the  State 
and  Federal  governments  under  a variety  of  grants  and  some  of  these  monies  must  be 
spent  on  particular  services.  In  addition,  the  Department  must  match  Federal  and  State 
dollars  with  a percentage  of  local  dollars  in  the  services  it  contracts  out.  For  example,  the 
Department  receives  $1,069,405  in  Title  III  C-l  funds  from  the  Federal  government. 
These  funds  are  broken  down  into  $96,172  for  administration  and  $973,233  for 
congregate  nutrition  programs.  The  Department  must  match  the  administration  Federal 
funding  by  25%  with  local  monies  and  must  match  the  congregate  nutrition  program’s 
Federal  funding  by  10.53%  with  local  monies.  This  means  that  the  Department  must 
match  the  Federal  funding  of  $1,069,405  in  Title  III  C-l  with  a match  of  $126,524  local 
dollars. 


Even  after  meeting  Federal  and  State  funding  and  local  matching  requirements,  the  OOA 
has  around  $10  million  dollars  of  discretionary  funding  (see  Finding  1-C)  that  it  can 
spend  on  services  other  than  the  Federal  and  State  mandated  programs.  The  OOA  can 
thus  allocate  a considerable  amount  of  funding  according  to  the  needs  of  seniors  and 
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adults  with  disabilities.  As  discussed  in  Finding  1-A,  the  needs  assessment  does  not 
analyze  and  show  what  services  should  be  funded  or  justify  the  funding  amounts  spent  on 
each  service.  The  Department  rather  summarizes  its  research  by  stating  that  housing, 
transportation,  and  home  and  community  based  services  are  the  top  three  identified 
needs.  However,  these  results  are  too  broad  for  purposes  of  allocating  the  OOA’s  SI  7 
million  to  various  services  and  does  not  justify  the  amounts  that  the  Department  should 
allocate  to  each  particular  service.  Because  of  these  shortcomings,  the  auditors  examined 
the  results  of  each  of  the  needs  assessment  studies  mentioned  in  the  Area  Plan  and  took  a 
tally  of  the  number  of  times  needs  were  identified.  The  tally  reflects  an  indicated  need 
for  more  or  better  quality  of  a service.  The  needs  listed  most  frequently  in  the  Area  Plan 
are  shown  below. 


Figure  2 


Frequency  of  Needs 


The  need  for  senior  and  disabled  adult  housing  is  prevalent — arising  in  the  needs 
assessment  described  above,  in  the  Fall  2002  need  assessment  studies,  and  in  discussion 
with  Department  staff.  Yet  the  00 A spends  less  than  1%  of  its  funding  on  housing. 
Transportation  was  the  next  most  identified  need  after  housing,  and  receives 
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approximately  4%  of  all  funding.  IHSS,  third  in  frequency,  is  funded  for  only  2%  of  all 
OOA  funds.  In  addition,  several  other  needs  were  identified  yet  receive  very  little 
funding  compared  to  funding  for  meals.  Community  Services,  and  Neighborhood 
Resource  Centers.  For  example,  the  tally  above  shows  that  the  need  for  case 
management  was  identified  as  many  times  as  meals,  yet  the  OOA  only  funds  its  Care 
Management  services  with  5%  of  its  funds  and  spends  around  46%  on  meals.  In  an 
another  example.  Information,  Referral  and  Assistance  (I,  R & A)  services  are  not 
frequently  cited  in  the  Department’s  needs  assessment,  yet  a significant  amount  of 
resources,  both  direct  and  contracted,  are  devoted  to  this  area. 

According  to  Department  staff  and  documentation  supplied  to  the  auditors,  funding 
allocation  for  particular  services  in  the  FY  2003-04  RFP  process  were  made  primarily 
based  on  prior  years’  allocations  rather  than  on  determinations  of  current  needs 
established  through  the  Department’s  needs  assessment  process.  In  simple  terms,  the 
level  of  funding  allocated  to  services  is  not  justified  in  the  Area  Plan  or  in  any  of  the 
Department’s  needs  assessment  studies.  The  Department  incorrectly  claimed  that  meals 
were  the  most  important  need  identified  in  the  Fall  2002  Needs  Assessment  when  it 
allocated  $552,1 10  of  the  $1.16  million  of  add-back  funding2  to  meals  programs. 
However,  the  Fall  2002  Needs  Assessment  did  not  conclude  that  meals  were  the  most 
important  need  but  rather  noted  that  the  primary  areas  of  need  were  affordable/accessible 
housing,  transportation/paratransit,  and  nutrition. 

The  City’s  Human  Services  Agency  does  spend  hundreds  of  millions  on  exactly  these 
needs — more  than  $75  million  on  IHSS  annually  including  $40  million  in  local  funds, 
and  more  than  $40  million,  of  which  over  $30  million  is  local  funds,  on  housing 
programs,  some  of  which  benefits  seniors  and  adults  with  disabilities.  As  the  primary 
planning  document  for  senior  services,  the  Area  Plan  should  lay  out  how  this  spending, 
and/or  the  work  of  other  agencies  and  programs,  is  meeting  the  population’s  needs  and 
thereby  justify  the  allocation  of  OOA  funds  to  certain  types  of  services.  Without  this 
analysis  there  is  no  assurance  that  the  OOA  is  funding  the  most  critical  needs  of  seniors 
and  adults  with  disabilities  in  San  Francisco.  The  Area  Plan  and  the  Department’s  needs 
assessment  activities  do  not  justify  the  current  allocation  of  the  OOA  resources  among 
the  29  different  services  provided. 


FINDING  1-C.  THE  OOA  OVERMATCHES  ITS  LOCAL  FUNDING  FOR 
FEDERAL  AND  STATE-MANDATED  PROGRAMS  BY  AN  ESTIMATED  $10 
MILLION.  THESE  DISCRETIONARY  LOCAL  DOLLARS  COULD  BE 
DIRECTED  TO  OTHER  SERVICES  FOR  SENIORS  AND  ADULTS  WITH 
DISABILITIES  OR  FOR  ANY  OTHER  PUBLIC  PURPOSE. 


For  FY  2004-05,  the  OOA  has  around  $1 1.5  million  in  local  funds  and  $5.8  million  in 
Federal  and  State  funds  that  are  dedicated  to  contract  services.  This  means  that  around 
one-third  of  the  OOA  funding  for  contracted  services  comes  from  the  Federal  and  State 
governments.  Federal  and  State  programs  require  local  matches  in  varying  percentages, 


2 The  Department  received  add-back  funding  from  the  Mayor  and  the  Board  of  Supervisors  for  its 
contracted  services  during  the  RFP  process  for  FY  2003-04.  The  specifics  of  the  add-back  funding  are 
discussed  in  Chapter  3 of  this  report. 
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but  the  Department  does  not  review  the  matching  requirements  for  each  of  the  programs 
in  detail,  instead,  it  matches  the  relevant  program  areas  with  approximately  $3  million  in 
local  funds,  constituting  a local  match  of  about  52%  overall— significantly  higher  than 
required  by  any  program.  The  Department  estimates  that  actual  minimum  matching 
requirements  are  around  $1.8  million  (32%  of  Federal  and  State  funding),  and  that  the 
“overmatch”  of  local  funds  is  around  $10  million. 

The  effect  of  the  $10  million  overmatch  is  that  the  OOA  allocates  a large  potion  of  its 
discretionary  funding  to  the  same  programs  it  receives  Federal  and  State  dollars  for, 
namely  meals  programs.  The  OOA  receives  around  $3.4  million  to  spend  on  meals  from 
the  Federal  and  State  government.  The  exact  amount  of  local  monies  the  OOA  must 
spend  on  meals  to  match  the  $3.4  million  is  undetermined  because  of  the  manner  in 
which  the  OOA  converts  grant  monies  to  specific  contracted  services.  However,  a 
reasonable  estimate  is  that  the  OOA  must  match  Federal  and  State  dollars  for  meals  by 
20%.  This  means  that  the  OOA  would  have  to  contribute  an  additional  $684,475  of  its 
local  monies  to  meals  besides  the  $3.4  million  it  receives  from  the  Federal  and  State 
governments.  The  estimated  total  amount  the  OOA  must  spend  on  meals  is  therefore 
$4. 1 million.  However,  the  OOA  spends  around  $8  million  on  meals — approximately 
46%  of  its  total  budget  and  33%  of  its  local  funds.  The  OOA  therefore  chooses  to  spend 
an  additional  $3.9  million  of  its  local  monies  on  meals  even  though  they  are  not  required 
to  do  so. 

At  a minimum,  the  Department  should  detail  the  amounts  of  local  monies  that  must  be 
spent  to  supplement  Federal  and  State  programs.  The  Department  does  not  have  an 
adequate  grasp  of  these  minimum  local  matching  requirements  per  program  and  instead 
overmatches  Federal  and  State  dollars  with  a large  portion  of  local  monies.  The  result  is 
that  the  OOA  is  supplementing  State  and  Federal  programs  with  more  monies  than 
required.  The  OOA  could  re-direct  approximately  $10  million  of  local  funds  to  meet 
other  critical  needs  of  seniors  and  adults  with  disabilities.  For  example,  at  a cost  of 
approximately  $1  million,  the  OOA  could  roughly  double  the  units  provided  for  some  of 
the  services  that  address  housing,  transportation  and  support  needs  such  as  grants-in-aid 
for  emergency  housing  assistance  ($40,000  to  provide  an  additional  1 00  grants),  group 
van  rides  ($680,000  for  an  additional  40,000  trips)  and  personal  care,  homemaker,  and 
chore  services  ($280,000  for  an  additional  17,100  hours  of  service).  Any  funding 
allocation  must  take  into  account  the  impact  on  current  services  and  the  limited  resources 
available  to  the  OOA.  Currently,  the  needs  assessment  activities  are  inadequate  at 
linking  resources  to  needs  and  the  Department  lacks  stated  priorities  for  allocating  OOA 
funding. 

RECOMMENDATIONS 

■ The  Department  should  use  the  needs  assessment  information  to  identify  specific 
needs  and  target  populations,  identify  current  resources  and  gaps  in  service, 
establish  clear  service  priorities,  set  goals  and  objectives,  and  allocate  the  OOA’s 
limited  funds  accordingly.  The  information  should  also  be  used  for  broader 
strategic  purposes  such  as  identifying  needed  legislative  changes  and  leveraging 
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funds.  The  Department  may  want  to  solicit  assistance  from  an  outside  consultant 
for  these  purposes. 

As  part  of  its  Area  Plan,  the  Department  should  develop  a more  comprehensive 
approach  to  assessing  the  types  and  level  of  services  being  provided  to  seniors 
and  adults  with  disabilities  by  other  organizations,  including  other  City 
departments.  Such  a process  should  specify  and  quantify  the  services  being 
provided  and  include  an  analysis  of  unmet  needs  so  that  the  Department  avoids 
duplication  and  efficiently  funds  needed  services. 

The  Department  should  review  other  county  area  plans  and  model  its  work  on 
some  of  the  best  practice  methodologies  described  in  the  audit  findings  above. 
Specifically,  the  Department  should  review  how  other  counties  identify  specific 
needs  and  target  populations,  identify  current  resources  and  gaps  in  service, 
establish  clear  service  priorities,  set  goals  and  objectives,  and  allocate  funding. 

The  Department  needs  to  justify  the  level  of  funding  allocated  to  its  contracted 
services  in  its  needs  assessment.  Specifically,  the  Area  Plan  should  describe  how 
the  needs  identified  through  the  needs  assessment  activities  are  met  by  the 
specific  services  the  Department  contracts  out. 

The  Department  needs  to  determine  the  minimum  matching  requirements  for  each 
of  the  Federal  and  State  monies  it  receives  and  spends  on  specific  services.  After 
the  Department  accomplishes  this  task,  the  Department  should  use  the 
discretionary  local  funding  available  to  fund  services  that  are  reflective  of  the 
needs  of  seniors  and  adults  with  disabilities  identified  in  the  Department’s  needs 
assessment  activities. 
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CHAPTER  2 

THE  DEPARTMENT  SHOULD  IMPROVE  ITS  CONTRACT 
MONITORING  PROCEDURES  AND  TOOLS  IN  ORDER  TO 
ADEQUATELY  CAPTURE  CONTRACTOR  PERFORMANCE  AND 
ENSURE  CONSISTENCY 


CHAPTER  SUMMARY 

The  Department  of  Aging  and  Adult  Services  (DAAS)  uses  a series  of  tools  to 

monitor  contracts,  such  as  checklists  and  units  of  service  reports,  and  has  six  staff 
and  one  manager  (currently  vacant)  for  this  purpose3.  Overall,  the  monitoring 
system  is  weak — assessment  and  follow-up  activities  are  inconsistent,  program 
staff  have  different  understandings  of  the  tools  and  requirements  that  should  be  enforced, 
and  management  oversight  is  minimal.  DAAS  does  have  some  contract  monitoring 
policies  and  procedures  in  place,  but  they  are  not  being  applied  in  a consistent  and 
uniform  manner.  Some  contractors  are  allowed  to  administer  their  own  contract 
monitoring  assessments,  while  others  have  the  assessments  administered  by  staff. 
Follow-up  is  inconsistent  even  where  monitoring  reports  clearly  indicate  a need  to 
address  findings.  Almost  all  of  the  Department’s  contract  monitoring  tools  are  designed 
to  capture  contractor  compliance  with  State  and  Federal  requirements  and  report  the 
quantities  of  service  expected.  Contractors  are  not  being  adequately  evaluated  on  the 
quality  of  the  services  and  insufficient  program-specific  data  is  captured  to  adequately 
assess  the  services  provided  for  particular  service  categories.  The  Department  of  Human 
Services’  Office  of  Contract  Management  (OCM)  assumption  of  responsibility  for  DAAS 
contract  administration  and  monitoring  will  not  by  itself  address  these  weaknesses. 

An  effective  contract  monitoring  system  should  consist  of  structures,  policies  and 
procedures  sufficient  to  ensure  that  the  objectives  of  a contract  are  accomplished,  vendors 
meet  their  responsibilities,  and  the  City’s  risk  is  mitigated.  An  adequate  contract 
monitoring  process  should  incorporate  the  following  elements: 

• Contract  monitoring  training  for  staff 
• Written  policies  and  procedures 
• Clearly  communicated  expectations  to  vendors 
• Organized  and  sufficiently  documented  contract  files 
• Regular  programmatic  reports  from  vendors 
• On-site  monitoring/visits 
• Funding  linked  to  satisfactory  performance 

• Use  of  incentives  for  good  performance  and  consequences  for  poor  performance 
• Measures  of  customer  satisfaction 


J Prior  to  consolidation,  contract  management  was  under  the  authority  of  its  Deputy  Director  of  Finance 
and  Administration,  who  has  since  retired.  Contract  management  duties  are  now  the  purview  of  the  Office 
of  Contracts  Management  of  the  Human  Services  Agency,  into  which  one  OOA  Program  Analyst  and  one 
OOA  Nutritionist  have  transferred. 
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Audit  staff  reviewed  the  Department’s  written  policies  and  procedures  for  contract 
monitoring  as  well  as  the  Department’s  contract  monitoring  files  to  determine  the  extent 
and  adequacy  of  the  Department’s  programmatic  monitoring.  Audit  staff  also  reviewed 
the  Department’s  contract  monitoring  training  for  staff  as  well  as  the  specific  staffing 
assignments  and  forms  used  for  monitoring.  In  addition,  audit  staff  examined  the  quality 
and  reliability  of  the  monitoring  tools  themselves  and  the  consistency  and  uniformity  of 
their  use  among  the  Department’s  Program  Analysts. 


FINDING  2-A.  CONTRACT  MONITORING  IS  INCONSISTENT  AND  WEAK. 
SOME  PROVIDERS  ARE  DOING  THEIR  OWN  ASSESSMENTS. 
MONITORING  TOOLS,  DOCUMENTATION  AND  FOLLOW-UP  ARE  NOT 
CONSISTENT.  TRAINING  AND  ROTATION  OF  PROGRAM  ANALYST 
STAFF  DO  NOT  OCCUR  AND  MANAGEMENT  OVERSIGHT  IS  MINIMAL. 


Four  manuals  are  available  to  DAAS  Program  Analysts  who  have  responsibility  for 
contract  monitoring  policies  and  procedures;  the  Policy  Memoranda  Manual  for  All 
Contractors  of  the  Office  on  Aging,  the  Linkages  Program  Manual,  the  Alzheimer’s  Day 
Care  Resource  Centers  Program  Manual  and  the  Office  On  Aging  (OOA)  Staff  Manual. 
Although  these  manuals  are  extensive,  they  lack  specific  references  to  contract 
monitoring  policies  and  procedures.  Key  contract  monitoring  tools  currently  utilized  by 
Department  staff,  such  as  the  Consumer  Satisfaction  Survey  and  a basic  report  from  the 
Department’s  on-line  tracking  system,  the  SF  GetCare  Variance  Report , are  not 
mentioned  in  any  of  the  procedural  manuals.  There  are  no  directions  in  any  of  the 
procedural  manuals  regarding  follow-up  procedures  or  documentation  requirements  for 
contract  monitoring. 

In  reviewing  the  contract  monitoring  processes  and  interviewing  staff,  the  audit  found 
that  staff  and  managers  do  not  have  an  adequate  grasp  of  the  monitoring  procedures  and 
requirements.  Program  Analysts  who  met  with  the  audit  team  initially  identified  the  SF 
GetCare  Variance  Reports , Annual  Baseline  Assessments,  and  Consumer  Satisfaction 
Surveys  as  the  only  three  mandatory  contract  monitoring  tools  used.  Subsequent  research 
determined  that  there  are  26  additional  forms  and  monitoring  tools  and  reports  in  use  for 
various  purposes  and  programs.  Some  of  these  tools  capture  the  same  information.  For  a 
complete  list  of  all  contract  monitoring  tools  see  Appendix  C. 

Audit  staff  evaluated  a sample  of  seventeen  contracts  in  order  to  ascertain  the  consistency 
and  completeness  of  monitoring  procedures.  Contractors  at  a high,  medium,  and  low 
level  of  funding  were  chosen  in  order  to  analyze  a broad  spectrum  of  service  providers. 
Contractor  assessment  files  were  reviewed  to  ensure  that  they  contained  documentation 
showing  that  the  three  mandatory  monitoring  tools  had  been  used  and  to  evaluate  the 
consistency  of  the  monitoring. 

The  sampled  files  were  relatively  complete,  with  SF  Get  Care  Variance  Reports  and 
Annual  Baseline  Assessments  on  file  for  all  contractors  examined,  and  Consumer 
Satisfaction  Surveys  on  file  for  thirteen  of  the  seventeen.  However,  overall  the 
monitoring  procedures  were  not  being  administered  or  used  consistently.  The  Annual 
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Baseline  Assessment  is  designed  to  ensure  that  funds  are  expended  in  compliance  with 
Federal,  State  and  local  regulations,  and  in  keeping  with  the  purposes  for  which  they 
were  awarded.  The  sample  of  Annual  Baseline  Assessments  found  that  some  were  self- 
administered  and  others  were  administered  by  Department  staff.  According  to  the 
Program  Analysts,  due  to  staffing  shortages,  “high  performing”  providers  are  allowed  to 
administer  their  own  Annual  Baseline  Assessments , while  others  have  their  assessments 
administered  by  Department  staff.  Approximately  1 1 out  of  45  contractors  were 
approved  for  self-assessments  for  FY  2004-05. 

Contractors  whose  Annual  Baseline  Assessment  results  indicate  the  need  for  follow-up 
are  not  receiving  consistent  treatment  by  the  Department.  According  to  the  Department, 
there  are  no  set  procedures  regarding  follow-up  methods  for  the  Annual  Baseline 
Assessment , and  no  regulations  or  procedures  regarding  follow-up  were  found  in  the 
Policy  Memoranda  Manual  for  All  Contractors  of  the  Office  on  Aging.  If  the  Annual 
Baseline  Assessments  resulted  in  some  sort  of  deficiency,  the  Program  Analysts  would 
issue  a compliance  memo  to  the  contractor,  but  the  ensuing  follow-up  is  largely  at  the 
discretion  of  the  individual  Program  Analyst.  Some  Program  Analysts  may  follow  up  in 
writing,  while  others  may  follow  up  verbally.  Among  the  files  evaluated,  there  were  files 
with  no  evidence  of  follow-up  even  though  the  Annual  Baseline  Assessment  indicated 
findings.  Some  files  also  lacked  compliance  verification  memos,  which  verify  that  the 
findings  detailed  in  the  Annual  Baseline  Assessment  have  been  resolved. 

The  weaknesses  in  the  Department’s  contract  monitoring  procedures  are  exacerbated  by  a 
lack  of  managerial  oversight  of  the  Program  Analysts,  lack  of  job-specific  training  for 
contract  monitoring,  and  lack  of  rotation  of  contract  monitoring  assignments  among 
Program  Analysts.  The  Department’s  Program  Manager  position  with  responsibility  for 
supervising  and  training  the  Program  Analysts  has  been  vacant  since  August  of  2004. 
Processes  that  were  already  inconsistent  when  the  Program  Manager  was  in  place  have 
become  more  so,  with  individual  Program  Analysts  utilizing  varied  approaches  to  the 
contract  monitoring  function. 

The  Department  also  lacks  structured  training  for  the  Program  Analysts  with  content  that 
addresses  contract  monitoring  policies  and  procedures.  Documentation  regarding  staff 
training  includes  many  classes  on  topics  pertinent  to  seniors  and  adults  with  disabilities, 
but  there  is  no  indication  that  there  were  or  are  any  training  sessions  regarding  the 
contract  monitoring  policies  and  procedures  or  the  duties  of  a Program  Analyst.  Lack  of 
rotation  of  contract  monitoring  assignments  among  Program  Analysts  has  also 
contributed  to  inconsistency  in  contract  monitoring.  Contractors  have  the  same  monitor 
from  year  to  year,  and  differences  in  contract  monitoring  approaches  have  emerged  and 
stayed  in  place.  Although  some  reassignments  occurred  in  November  2004  as  a result  of 
the  DAAS/DHS  merger  that  moved  some  Program  Analysts  to  DHS,  there  is  currently  no 
regularly  scheduled  rotation  of  contract  monitoring  assignments  for  the  Program 
Analysts.  Department  staff  stated  that  there  was  discussion  regarding  rotation  of 
assignments.  However,  Program  Analysts  were  resistant  and  the  idea  was  abandoned. 

Feedback  obtained  from  the  auditor’s  focus  group  with  service  providers  further 
substantiated  the  lack  of  consistency  and  uniformity  in  contract  monitoring  procedures. 


Office  of  the  Controller 


-Page  25- 


May  2,  2005 


A significant  number  of  the  contract  providers  interviewed  stated  that  the  contract 
monitoring  methods  vary  greatly  among  the  individual  Program  Analysts.  Permanent 
contract  monitoring  assignments,  combined  with  procedural  inconsistencies,  has  created 
a perception  among  contractors  of  favoritism  by  the  Department. 


Taken  together,  the  weaknesses  discussed  above  may  cause  the  Department  to  fail  to 
identify  under-performing  contractors  or  those  at  risk  of  not  being  able  to  fulfill  their 
contract  obligations,  thereby  affecting  the  services  received  by  seniors  and  adults  with 
disabilities. 

RECOMMENDATIONS 

■ DAAS  should  develop  a current  policies  and  procedures  manual  for  contract 
monitoring  that  provides  detailed  information  regarding  contract  monitoring 
processes,  guidelines  regarding  the  use  of  specific  contract  monitoring  tools,  a 
schedule  specifying  the  frequency  of  contract  monitoring  procedures, 
identification  of  mandatory  contract  monitoring  procedures  that  distinguishes 
between  compliance  monitoring  for  State  and  Federal  regulations  and 
performance  monitoring,  methodologies  for  testing,  requirements  for  staff 
rotation,  review  procedures  for  management,  comprehensive  follow-up 
procedures  for  any  deficiencies  resulting  from  monitoring  and  the  types  of 
supporting  documentation  that  should  be  retained. 

■ DAAS  should  fill  the  vacant  Program  Manager  position  to  provide  the  Program 
Analysts  with  sufficient  oversight,  guidance  and  training. 

■ DAAS  should  rotate  contract  monitoring  assignments  among  the  Program 
Analysts.  Regularly  scheduled  rotation  of  contract  monitoring  assignments  would 
also  allow  Program  Analysts  to  become  acquainted  with  a wider  variety  of  DAAS 
funded  contractors  and  service  delivery  issues. 

■ DAAS  management  should  implement  a training  program  on  contract  monitoring 
for  the  Program  Analysts  utilizing  the  elements  discussed  above  (policies  and 
procedures,  monitoring  tools,  compliance  and  programmatic  approaches). 

■ DAAS  management  should  emphasize  the  importance  of  consistent  and  uniform 
contract  monitoring  procedures  to  the  Program  Analysts  and  periodically  sample 
contract  files  to  assess  whether  contracts  are  being  monitored  appropriately. 


FINDING  2-B.  DAAS  CONTRACT  MONITORING  TOOLS  EMPHASIZE  UNITS 
OF  SERVICE  AND  COMPLIANCE  DATA  BUT  ARE  NOT  DESIGNED  TO 
ADEQUATELY  CAPTURE  CONTRACTOR  PERFORMANCE. 


In  order  to  adequately  evaluate  contractor  performance,  the  Department  needs  to  collect 
both  sufficient  qualitative  data  to  determine  whether  contractors  are  providing  high 
quality  service  to  their  clients  and  sufficient  quantitative  data  to  determine  if  the 
contractors  are  meeting  the  units  of  service  requirements  stipulated  in  their  contracts.  For 
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the  purposes  of  this  audit,  qualitative  data  refers  to  information  that  reflects  the  quality  of 
services  provided  by  the  individual  contractors  (such  as  the  language  capabilities  of  the 
staff  or  the  comfort  level  of  the  facilities).  Quantitative  data  refers  to  numerical  data 
gauging  the  quantity  or  amount  of  services  provided  (such  as  units  of  service  data). 
Compliance  data  refers  to  information  that  verifies  conformity  with  a given  set  of  rules, 
laws  or  regulations  (such  as  the  Federal  regulations  for  nutrition  programs). 

The  three  mandatory  contract  monitoring  tools  identified  by  staff  are  designed  to  capture 
quantitative  and  compliance  oriented  data  and  collect  little  information  about  the  quality 
of  the  services  being  provided.  The  Annual  Baseline  Assessment  form  contains  six  out  of 
seventy  questions/criteria  geared  towards  collecting  qualitative  data,  with  the  remainder 
collecting  compliance  related  data.  Likewise,  the  Annual  Consumer  Satisfaction  Survey 
contains  five  generic  questions  about  satisfaction  with  the  contractor  but  does  not  capture 
information  about  customer  satisfaction  with  the  quality  of  the  specific  services  being 
provided.  This  is  important  because  contractors  often  provide  multiple  types  of  services 
and  satisfaction  with  particular  services  will  vary.  Finally,  the  SF  GetCare  Variance 
Reports  only  show  the  provider’s  adherence  to  the  units  of  service  standards  stated  in 
their  contract. 

The  lack  of  qualitative  information  in  the  Department’s  contract  monitoring  tools  can 
directly  impact  the  services  consumers  receive.  The  Department’s  documentation  does 
not  distinguish  between  high  quality  service  providers  and  low  quality  service  providers. 
For  example,  the  Department  contracts  with  29  separate  contractors  for  Community 
Services.  The  specific  services  include  activity  scheduling,  translation,  and  social 
services,  with  the  units  of  service  measured  in  hours.  The  Department  tracks  whether  or 
not  the  29  agencies  are  providing  the  total  hours  of  services  contracted  for  but  does  not 
monitor  quality  of  these  specific  services.  The  Department  could  monitor  the  types  and 
quality  of  the  social  services,  the  effectiveness  of  the  translation  services,  and  the  quality 
of  the  counseling  services  being  provided.  Consequently,  the  Department  has  no  basis 
for  comparing  the  services  provided  by  these  29  separate  contractors  and  cannot  use 
monitoring  information  to  determine  how  to  distribute  its  funding  or  whether  to  provide 
technical  assistance  to  improve  quality. 

DAAS  contract  monitoring  tools  should  also  gather  sufficient  program-specific  data  to 
accurately  assess  the  service  provided.  Although  the  Annual  Baseline  Assessment  was 
designed  using  California  Department  of  Aging  standards,  it  is  generic  and  is  being 
utilized  to  measure  a broad  range  of  disparate  services,  from  transportation  to  legal 
services.  Similarly,  the  Consumer  Satisfaction  Survey  used  by  the  Department,  which 
should  theoretically  provide  measures  of  quality,  is  designed  primarily  with  yes/no 
questions  and  does  not  capture  information  about  the  specific  services  being  provided. 

The  overall  effect  is  that  quantitative  and  compliance-oriented  analysis  is  the  focal  point 
of  DAAS’  contract  monitoring  processes  and  service  providers  are  not  being  adequately 
evaluated  on  the  quality  of  the  services  they  are  contracted  to  provide. 

Monitoring  tools  are  general  in  nature  and  do  not  capture  sufficient  program-specific  data 
or  service  issues  with  the  exception  of  nutrition  programs  that  are  monitored  by 
Nutritionists  under  State  regulations.  While  monitoring  tools  for  nutrition  programs 
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include  qualitative  standards,  as  a whole,  the  Department’s  monitoring  tools  are  deficient 
and  the  three  main  contract  monitoring  tools  identified  by  staff  are  designed  to  capture 
quantitative  and  compliance  oriented  data  and  collect  little  information  about  the  quality 
of  the  services  being  provided.  The  lack  of  qualitative  and  program-specific  data  makes 
it  difficult  to  determine  whether  contractors  are  providing  seniors  or  adults  with 
disabilities  with  good  quality  service.  In  some  cases,  gaps  in  information  are  being  filled 
with  separate  monitoring  processes  and  tools.  Staff  effort  is  duplicative  and  some 
contractors  are  required  to  participate  repeatedly  in  segmented  monitoring  activities. 

RECOMMENDATIONS 

■ The  Department  should  consider  implementing  new  contract  monitoring  tools  or 
changing  existing  contract  monitoring  tools  to  incorporate  qualitative  assessments 
of  services  provided  by  contractors.  For  example,  monitoring  tools  should 
measure  the  quality  of  the  health  services,  the  effectiveness  of  the  translation 
services,  and  the  quality  of  the  counseling  services  being  provided  by  the  29 
Community  Services  contractors  and  not  just  the  number  of  hours  of  activity 
scheduling,  number  of  hours  of  translation,  and  number  of  social  services  hours 
provided.  Examples  of  other  qualitative  assessment  criteria  applicable  to  a variety 
of  services  are: 

o Tangibles:  The  physical  facilities,  equipment,  appearance  of  personnel 
and  the  presence  of  other  participants. 

o Reliability:  The  ability  to  perform  the  promised  service  dependably  and 
accurately. 

o Responsiveness:  Willingness  to  help  participants  and  to  provide  prompt 
service. 

o Assurance:  Knowledge  and  courtesy  of  staff  and  their  ability  to  convey 
trust  and  confidence. 

o Empathy:  Caring  individualized  attention  to  participants. 

■ The  Department  should  consider  implementing  new  contract  monitoring  tools  or 
changing  existing  contract  monitoring  tools  to  incorporate  program-specific 
measures  for  the  various  specialized  services.  The  new  or  revised  contract 
monitoring  tools  should  also  include  measures  of  the  services  being  provided 
rather  than  relying  on  overall  satisfaction  with  a contractor. 


FINDING  2-C.  SF  GETCARE  REPORTS  ARE  UNRELIABLE  AND 
INACCURATE,  MAKING  IT  DIFFICULT  FOR  THE  DEPARTMENT  TO 
CONDUCT  ACCURATE  ASSESSMENTS  OF  SERVICES  PROVIDED  BY 
CONTRACTORS. 


The  Department  uses  a web-based  information  system  called  SF  GetCare  to  track  units  of 
service  provided  by  contractors.  This  system  provides  the  format  and  reporting 
information  required  by  the  California  Department  of  Aging  (CDA).  The  system  is 
accessible  online  to  providers  under  contract  with  DAAS  to  record,  track,  manage  and 
report  information  on  consumers  and  the  services  they  receive.  Department  staff  creates 
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a report  from  the  SF  GetCare  system  called  the  SF  GetCare  Variance  Report,  which  is 
designed  to  measure  disparities  between  the  units  of  service  (UOS)  that  the  contractors 
are  obligated  to  provide  and  the  actual  units  of  service  delivered. 

In  the  course  of  the  audit,  incorrect  information  was  found  in  a randomly  selected  SF 
GetCare  Variance  Report  and  upon  further  inquiry,  numerous  sources,  including 
Program  Analysts,  contractors  and  management  staff,  confirmed  that  the  information  in 
the  SF  GetCare  Variance  Reports  is  often  faulty.  Department  staff  also  provided  the 
audit  team  with  spreadsheets  that  some  Program  Analysts  use  to  track  variances  in  units 
of  service  among  the  contract  providers.  Staff  members  recreate  these  reports  in  addition 
to  using  the  SF  GetCare  Variance  Reports  because  of  inaccurate  or  unreliable  data  on  the 
part  of  the  SF  GetCare  system.  Feedback  from  contractors  during  the  audit  focus  group 
process  indicated  that  contractors  also  maintain  separate  tracking  documents.  Although 
the  audit  did  not  test  whether  the  inaccurate  SF  GetCare  service  data  is  caused  by  system 
problems  or  by  errors  on  the  part  of  contractors  inputting  the  data,  there  is  some 
indication  that  the  problem  is  due  to  data  management  and  reporting  problems  in  the 
system  itself. 

The  unreliable  units  of  service  data  from  SF  GetCare  make  it  difficult  to  gauge  whether 
providers  are  complying  with  the  terms  of  their  contracts.  DAAS  may  fail  to  identify 
duplication  of  effort,  gaps  in  services,  and  under-performing  providers.  Specifically,  the 
Department  may  pay  for  services  that  are  not  being  rendered,  or,  alternatively,  fail  to 
reimburse  contractors  for  their  cost  of  service. 

As  of  the  writing  of  this  report,  the  Department  is  in  the  process  of  making  changes  to  SF 
GetCare.  The  Department  is  ending  its  contract  with  the  current  service  provider,  RTZ 
Associates,  and  is  developing  a contract  with  a new  service  provider.  The  information 
and  referral  component  that  is  publicly  available  on  the  web  will  be  maintained  by  a new 
contractor  and  will  be  launched  in  the  spring  of  2005.  The  component  which  is  used  by 
contractors  to  enter  service  data  and  by  Program  Analysts  to  monitor  contractors  will  be 
brought  in-house  and  maintained  directly  by  the  Department’s  own  information 
technology  staff  and  will  transition  the  following  fiscal  year.  The  Department  anticipates 
that  these  changes  will  solve  the  problems  they  have  been  experiencing  with  SF  GetCare. 

City  financial  records  show  periodic  payments  to  RTZ  totaling  $2.6  million  from  FY 
2000-01  to  FY  2004-05  for  both  development  and  maintenance  work  done  through  FY 
2003-04,  and  the  Department  has  not  yet  been  invoiced  by  RTZ  for  any  costs  incurred  in 
the  current  fiscal  year.  Some  of  the  funding  for  SF  GetCare  was  provided  through  grants 
restricted  to  that  purpose,  and  some  through  the  General  Fund.  According  to  the  HSA, 
the  new  reporting  tools  will  have  an  estimated  set-up/development  of  $50,000  and  service 
costs  estimated  at  $42,000  annually,  plus  City  staff  costs. 

RECOMMENDATIONS 

■ As  noted  above,  the  Department  is  currently  working  to  replace  SF  GetCare  with 
tools  that  are  more  accurate,  reliable,  can  be  used  by  contractors,  and  do  not 
require  duplication  of  efforts  by  the  Program  Analysts  to  determine  whether 
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contractors  are  complying  with  the  terms  of  their  contracts.  The  Department 
should  establish  a regular  program  of  testing  the  accuracy  of  service  data  and 
reports. 


Given  the  cost  and  performance  problems  identified  with  SF  GetCare,  the 
Department  should  also  evaluate  the  contract  with  RTZ  Associates  and  determine 
whether  its  service  expectations  have  been  met. 


FINDING  2-D.  THE  DEPARTMENT  OF  HUMAN  SERVICES’  OFFICE  OF 
CONTRACT  MANAGEMENT  CONTRACT  MONITORING  PROCEDURES 
WILL  NOT  ABATE  THE  ISSUES  IDENTIFIED  WITH  DAAS  PROGRAM 
MONITORING. 


As  discussed  in  the  introduction  to  this  report,  a reorganization  has  placed  the 
Department  of  Aging  and  Adult  Services  and  the  Department  of  Human  Services  (DHS) 
together  under  the  Human  Services  Agency  (HSA)  umbrella.  As  a result,  DHS’  Office  of 
Contract  Management  (OCM)  is  assuming  responsibility  for  the  administration  and 
monitoring  of  DAAS  contracts.  This  transition  was  occurring  as  the  audit  began  and 
continues  to  take  shape  as  of  the  writing  of  this  report.  As  currently  planned,  OCM  staff 
will  be  assuming  the  administration,  including  the  request  for  proposal  process,  and  fiscal 
monitoring  of  DAAS  contracts.  Program  monitoring  will  continue  to  be  performed  by 
DAAS  Program  Analysts. 

The  auditors  examined  OCM’s  fiscal  monitoring  procedures  to  determine  if  their  internal 
controls  over  contract  monitoring  could  be  applied  to  DAAS  contracts  in  order  to 
mitigate  the  deficiencies  identified  in  this  report.  In  summary,  OCM  does  conduct  fiscal 
monitoring  and  site  visits,  but  it  does  not  retain  evidence  of  monitoring  in  contract  files 
and  also  lacks  written  policies  and  procedures  for  administration  of  contracts.  In  order  to 
abate  the  problems  identified  in  this  report  OCM  will  need  to  address  the  specific 
problems  with  DAAS  contract  monitoring  rather  than  relying  solely  on  their  own 
procedures. 

The  Office  of  Contract  Management  Lacks 
Adequate  Written  Policies  and  Procedures  for 
Administration  and  Monitoring  of  Contracts 

The  audit  staff  requested  DHS’  policies  and  procedures  for  contract  monitoring  and  were 
provided  with  a half-page  document,  “Site  Visit  and  Fiscal  Monitoring,”  which  states  that 
the  purpose  of  the  site  visit  is  to  verify  that  the  contractor’s  fiscal  practices  are  proper  and 
that  contract  and  fiscal  compliance  are  adequate  and  the  “HSA  OCM  Site  Visit  Form” 
which  lists  the  contract  and  fiscal  compliance  verification  steps  the  contract  manager 
should  take  both  prior  to  and  during  a site  visit.  An  additional  document  is  a “Division  of 
Labor”  form,  which  describes  the  respective  duties  of  the  OCM  contract  managers  for 
fiscal  monitoring  and  the  DHS  division  program  managers  for  program  monitoring. 
According  to  the  OCM  Director,  other  than  the  documents  described,  there  are  currently 
no  comprehensive  policies  and  procedures  for  contract  administration  including  contract 
monitoring.  However,  the  Director  indicated  he  is  currently  working  to  develop  a 
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policies  and  procedures  manual  that  will  address  all  aspects  of  the  contracting  process. 
There  is  no  department-wide  set  of  policies  and  procedures  for  programmatic  monitoring, 
however,  divisions  within  DHS  do  have  their  own  policies  and  procedures  that  vary 
depending  on  the  types  of  contracts  they  have. 

The  Office  of  Contract  Management  Generally 
Conducts  Site  Visits  but  Does  Not  Retain 
Evidence  of  Monitoring  in  Contract  Files 

OCM’s  objective  is  to  conduct  a fiscal  monitoring  site  visit  once  a year  for  its 
contractors.  The  audit  selected  a sample  of  DHS  contract  files  to  determine  whether 
there  was  sufficient  evidence  that  monitoring  was  occurring  and,  if  so,  whether  the 
monitoring  appeared  adequate  to  ensure  that  contractors  were  complying  with 
requirements  and  whether  funds  were  being  spent  appropriately.  Nineteen  files  were 
reviewed  from  three  contract  areas:  Family  and  Children’s  Services,  Housing,  and 
Welfare  to  Work.  The  sampled  files  were  evaluated  for  whether  they  contained  some  or 
all  of  the  following: 

■ Engagement  letters  showing  the  dates  of  the  planned  monitoring; 

■ Follow-up  or  compliance  letters  describing  any  problems  found  and  what  was 
needed  to  resolve  them; 

■ Site  Visit  Forms; 

■ Contract  manager  notes  or  copies  of  any  documents  that  were  reviewed;  and 

■ Monitoring  Report  Summaries  that  accompany  the  contract  renewal  memo  to  the 
Human  Services  Commission. 

Our  review  found  that  while  most  contract  files  had  evidence  of  monitoring  activities, 
such  as  correspondence  from  the  Contract  Manager  or  hand-written  notes  made  during 
field  visits,  overall  the  evidence  in  the  files  was  not  sufficient  to  conclude  that  the 
monitoring  activities  were  consistent  or  adequate.  In  some  cases,  there  was  no  evidence 
that  either  fiscal  or  program  monitoring  had  occurred.  In  none  of  the  files  we  reviewed 
were  there  copies  of  documents  tested  or  reviewed  or  documentation  of  testing 
procedures  that  occurred.  None  of  the  files  contained  a completed  “HSA  OCM  Site  Visit 
Form.” 

For  performance  monitoring,  which  is  conducted  by  Program  Managers  in  the  various 
DHS  divisions,  the  only  evidence  that  monitoring  had  occurred  was  in  the  Monitoring 
Report  Summary  that  accompanies  contract  renewal  requests  to  the  Human  Services 
Commission.  Some  Monitoring  Report  Summaries  as  provided  to  the  Commission 
included  statements  about  the  results  of  performance,  as  well  as  fiscal  monitoring 
information.  In  some  cases,  there  was  evidence  in  the  Monitoring  Report  Summaries  that 
fiscal  monitoring  had  occurred  but  there  was  no  evidence  performance  monitoring  had 
occurred. 
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RECOMMENDATIONS 


OCM  should  develop  a policies  and  procedures  manual  for  contract  monitoring. 
As  discussed  in  the  recommendations  under  Finding  2-A,  such  a manual  should 
provide  detailed  information  regarding  how  fiscal  and  program  monitoring  of 
DHS  contracts  should  be  conducted,  including  monitoring  frequency,  the  types  of 
testing  that  should  be  performed  during  a monitoring  visit,  and  the  types  of 
supporting  documentation  that  should  be  retained.  To  the  extent  that  existing 
policies  and  procedures  used  by  DA  AS  are  applicable,  they  should  be 
incorporated  into  the  new  manual. 

OCM  should  also  develop  a site  visit  summary  form  (or  modify  the  existing  site 
visit  form)  to  provide  information  on  fiscal  and  programmatic  monitoring 
activities,  results,  and  corrective  actions  taken  and  to  serve  as  a basis  for  the 
report  that  accompanies  contract  renewal  requests  to  the  Human  Services 
Commission. 
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CHAPTER 3 

THE  DEPARTMENT  SHOULD  REVISE  THE  RFP  PROCESS  SO 
THAT  IT  IS  COMPETITIVE  AND  BASED  ON  OBJECTIVE 
CRITERIA  THAT  REFLECT  THE  NEEDS  IDENTIFIED  THROUGH 
THE  NEEDS  ASSESSMENT  PROCESS 


CHAPTER  SUMMARY 

The  Office  On  Aging  (OOA),  formerly  the  Commission  on  Aging  (COA),  is  the 

designated  Area  Agency  on  Aging  for  San  Francisco.  The  OOA  is  responsible  for 
implementing  the  mandates  of  the  Older  Americans  Act,  and  in  that  capacity,  it 
serves  as  the  planning,  advocacy,  service  coordination  and  systems  development 
body  for  services  for  older  persons.  The  Older  Americans  Act  requires  that  all  Area 
Agencies  on  Aging  conducting  RFPs  have  an  “open  and  competitive  process”  and  ensure 
that  “all  applicants  are  treated  equitably.”  The  expectation  is  that  the  Department 
establish  funding  guidelines  that  support  an  open  and  competitive  RFP  process.  Once 
established,  the  Department  should  make  every  effort  to  meet  those  guidelines. 

The  California  Code  of  Regulations  states  that  contract  awards  shall  be  limited  to  a one- 
year  period  but  at  the  discretion  of  the  OOA  contracts  may  be  renegotiated  up  to  a 
maximum  of  three  additional  one-year  periods  before  undergoing  an  RFP  process.  San 
Francisco’s  Department  of  Aging  and  Adult  Services  (DAAS)  renegotiates  contracts  with 
existing  service  providers  every  year  and  conducts  an  RFP  for  most  of  its  services  every 
three  years.  Around  85%  of  the  OOA  funds  are  in  the  same  RFP  cycle  with  the  exception 
of  the  Family  Caregiver  Support  Program,  Neighborhood  Resource  Centers  for  Seniors 
and  Adults  with  Disabilities,  and  Paratransit  services.  The  most  recent  RFP  process  took 
place  from  February  2003  to  July  2003  and  the  terms  and  provisions  of  the  contracts 
awarded  were  for  July  1,  2003,  through  June  30,  2004,  with  renewal  options  for  FY 
2004-05  and  FY  2005-06. 

The  Department’s  RFP  process  consists  of  two  main  parts;  determining  the  amounts  of 
funding  for  each  of  25  service  categories  and  selecting  bidders  for  funding  within  each 
category.  The  RFP  document  created  for  the  FY  2003-2004  process  was  reviewed  at  four 
separate  public  meetings  before  being  approved  and  issued.  Once  issued,  bids  are 
received,  scored,  and  ranked  by  the  Department.  The  Department’s  initial 
recommendations  to  fund  specific  bidders  was  presented  at  a meeting  of  the 
Department’s  Advisory  Council,  and  was  followed  by  two  other  meetings  that  provided 
feedback  regarding  the  Department’s  recommendations.  Funding  awards  for  bidders 
changed  multiple  times  during  these  meetings.  The  Commission  made  its  final  approvals 
on  June  4,  2003  and  considered  appeals  at  a Commission  hearing  on  June  11,  2003. 
Following  this  process  including  the  appeals  hearing,  the  Department  distributed  an 
additional  $1.16  million  in  “add-back”  funding  received  from  the  Mayor  and  Board  of 
Supervisors.  The  timeline  below  illustrates  the  allocation  process. 
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Determining  RFP 
Amounts 


Selecting 

Contractors 


Add-Back 

Funding 


2/26/2003 

DAAS  Finance  Committee 
Review  of  RFP 


2/13/2003 
DAAS  Finance  Committi 
Review  of  RFP 


2/19/2003 

DAAS  Advisory  Council 
Review  of  RFP  & DAAS 
Comission  on  Aging 
Review  of  RFP  3/5/2003 

Final  Commission 
On  Aging  Review 
Of  RFP 


6/13/2003 

Public  Meeting  to  Discuss 
Add-Back  Funding 


Commission  Meeting 
Approval  of  Add-Back 
Funding 


The  audit  found  that  consistent  and  objective  criteria  were  not  used  to  select  all 
contractors  or  allocate  funding  amounts.  The  Department  did  not  follow  the  RFP  rules  or 
funding  guidelines  it  established  and  the  outcomes  were  frequently  not  based  on  the  RFP 
scores.  The  additional  $1.16  million  in  add-back  funding  distributed  by  the  Department 
was  largely  allocated  in  a manner  not  reflective  of  either  the  RFP  scores  or  prior 
Commission  award  decisions.  For  example,  bidders  received  add-back  funding  even 
though  they  were  not  funded  in  the  RFP  process.  Some  bidders  did  not  receive  any 
funding  despite  the  fact  that  they  scored  higher  than  other  bidders  receiving  funding  in 
the  same  service  categories.  In  addition,  some  contractors  were  awarded  a greater 
percentage  of  the  amount  they  requested  while  other  bidders  with  higher  scores  were 
awarded  a smaller  percentage  of  the  amount  they  requested.  The  Department  used 
several  justifications  for  distributing  monies  to  particular  contractors  and  for  the  amounts 
distributed,  however,  these  justifications  were  not  consistent,  were  not  indicative  of  a fair 
and  competitive  process,  and  appear  to  have  been  influenced  by  advocacy  efforts  of  the 
bidders.  The  audit  also  found  that  the  Department  receives  revenue  from  the  State  at 
different  times  during  the  year  and  the  total  amounts  tend  to  fluctuate.  The  manner  in 
which  the  Department  funnels  these  changes  in  revenue  to  contractors  is  subject  to  the 
same  weakness  identified  in  the  RFP  process.  Some  of  the  exceptions  that  will  be 
discussed  are  summarized  in  Table  2 below.  Details  of  all  exceptions  to  the  RFP  scores 
and  rankings  noted  are  in  Appendix  D. 
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Table  2 


Exceptions 

Number  of  bids 

Number  of  bids  initially  recommended  for  funding  that  scored  under  70 
points4. 

8 

Number  of  bids  awarded  funding  by  the  Commission  during  RFP  process  that 
scored  under  70  points5. 

11 

Number  of  bids  ultimately  receiving  funding  in  FY  2003-04  that  scored  under 
70  points6. 

16 

Number  of  bids  receiving  funding  in  FY  2003-04  that  were  not  approved  for 
funding  by  the  Commission  during  the  RFP  Process7. 

6 

Number  of  bids  that  did  not  ultimately  receive  funding  even  though  they  scored 
& ranked  higher  than  other  bidders  that  received  funding8. 

4 

FINDING  3-A.  CONSISTENT  AND  OBJECTIVE  CRITERIA  WERE  NOT  USED 
TO  SELECT  ALL  CONTRACTORS  AND  TO  SET  CONTRACTOR  FUNDING 
AMOUNTS  AND  RESULTED  IN  DAAS  AWARDING  A CONTRACT  TO  A 
POORLY  QUALIFIED  SERVICE  PROVIDER. 


The  Department  established  several  rules  regarding  the  RFP  application  procedures, 
evaluation  criteria,  award  process,  budget  requirements,  contract  requirements,  and  audit 
requirements.  The  audit  review  of  the  evaluation  criteria  and  award  process  found  that 
the  Department  did  not  adhere  to  the  funding  rule  it  established  governing  the  RFP 
process.  In  addition,  the  Department  established  five  funding  guidelines  for  selecting 
contractors.  However,  these  guidelines  were  not  communicated  in  the  RFP  packet  given 
to  bidders  and  most  of  these  guidelines  were  not  followed  in  the  Department’s  award 
recommendations  or  the  Commission’s  funding  decisions.  The  one  guideline  that 
appears  to  have  been  consistently  followed,  however,  seems  to  run  contrary  to  the  intent 
of  a fair  and  competitive  RFP  process.  As  a result  of  the  Department  not  adhering  to  the 
funding  rule,  not  following  most  of  its  funding  guidelines,  and  that  the  only  funding 
guideline  followed  was  uncompetitive  and  contrary  to  the  intent  of  a fair  and  competitive 
RFP  process,  the  audit  concludes  that  objective  criteria  was  not  used  to  select  all 
contractors  or  set  contractor  funding  amounts. 


4 The  Department’s  first  funding  recommendations  discussed  at  the  Advisory  Council  Meeting,  5/21/03. 

5 As  of  the  end  of  the  Commission  Appeals  decisions,  6/1 1/03. 

6 Actual  amounts  in  FY  2003-04. 

7 As  of  the  end  of  the  Commission  Appeals  decisions,  6/1 1/03. 

8 As  of  the  end  of  the  July  2,  2003  COA  Add-Back  Meeting. 
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The  Department  Did  IXot  Adhere  to  the  Rule 
It  Established  Governing  the  RFP  Process 

The  RFP  application  packet  given  to  prospective  bidders  specifically  stated  in  bold  letters 
that  “ Applicants  receiving  a score  of  70  or  less  will  not  be  considered  for  funding.  ” As 
noted  above,  bidders  that  scored  under  70  points  were  not  only  considered  for  funding  in 
the  Department’s  deliberations,  but  were  also  recommended  and  ultimately  ended  up 
receiving  funding.  The  Department’s  initial  funding  recommendation  taken  to  the  DAAS 
Advisory  Council  was  to  fund  eight  bids  that  scored  under  70  points.  Some  of  these  bids 
were  recommended  for  funding  over  other  bids  that  scored  over  70  points.  After  all  the 
funding  award  changes  that  occurred  during  the  RFP  process  and  add-back  funding,  16 
out  of  the  20  bids  that  scored  under  70  were  funded.  See  Appendix  D for  full  details. 

When  the  Department  presented  its  first  award  recommendations  to  the  DAAS  Advisory 
Council  on  May  21, 2003,  they  included  guidelines  that  contradicted  those  initially 
communicated  to  bidders.  The  funding  guidelines  also  included  a disclaimer  that  stated 
that  these  guidelines  might  be  in  tension  with  one  another  and  that  the  results  of  the  RFP 
are  only  advisory  to  the  Commission  on  Aging.  These  funding  guidelines  stated: 

“To  the  extent  possible  the  following  recommendation  guidelines  will  be  utilized: 

1.  Proposals  scoring  under  70  points  should  not  be  funded. 

2.  Bids  should  not  be  funded  for  more  dollars  than  requested. 

3.  Services  should  be  available  in  underserved  geographical  areas. 

4.  Services  should  be  assured  for  targeted  populations. 

5.  Awards  should  approximate  current  service. 

Note:  For  some  recommendations  these  guidelines  support  and  function  well 

together.  In  other  instances  the  guidelines  are  in  tension  with  one  another,  e.g.,  low 
rating  but  service  is  in  underserved  geographical  area.  California  Department  of 
Aging  (CDA)  Regulations  for  Requests  for  Proposal,  Title  22,  Section  7358,  Item  C 
states,  ‘The  recommendations  of  the  Bid/Proposal  Evaluation  Panel  are  advisory  to 
the  AAA  decision-making  body  which  shall  be  responsible  for  all  award  decisions.’” 

It  seems  that  the  intent  of  the  changed  guidelines  and  disclaimer  in  this  document  was,  in 
part,  to  justify  funding  recommendations  that  differed  from  the  original  guidelines.  On 
this  issue,  the  auditors  note  that  that  the  CDA  regulation  quoted  by  the  Department 
continues  with:  “If  award  decisions  are  based  upon  factors  other  than  the  evaluation 
criteria  contained  in  the  RFP,  a full  justification  for  the  decision  shall  be  documented.” 
Title  22,  Section  7358,  Item  C 

The  Department  gave  some  justifications  for  funding  the  16  bids  out  of  20  that  scored 
under  70  points  during  the  RFP  and  add-back  meetings.  The  justifications  were  based 
upon  factors  other  than  the  evaluation  criteria  contained  in  the  RFP  and  were  incomplete, 
inconsistent  and  not  applied  equally.  For  example,  some  of  the  bidders  were  restored  to 
their  FY  2002-03  levels  and  some  were  pro-rated  to  a portion  of  their  previous  funding 
level  without  regard  to  the  RFP  scores.  One  bidder  that  scored  under  70  points  received 
funding  in  order  to  retain  staff  even  though  they  were  not  awarded  any  funding  as  a result 
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of  the  RFP  process  itself.  The  specific  reasons  given  for  funding  based  on  factors  other 
than  the  evaluation  criteria  are  described  in  detail  in  Appendix  D. 

The  significance  of  awarding  funds  based  upon  factors  other  than  the  evaluation  criteria 
contained  in  the  RFP  is  not  that  bidders  that  score  under  70  points  should  never  be 
awarded  funding.  In  some  cases,  there  may  be  good  reason  to  fund  bidders  that  score 
under  70  points.  However,  the  Department  did  not  communicate  its  funding  guidelines 
and  when  it  did,  the  guidelines  contradicted  what  was  initially  told  to  contractors. 

The  immediate  effect  of  the  Department  not  following  the  RFP  rule  was  widespread 
discontent  with  the  process  and  several  appeals  of  the  funding  decisions.  This  discontent 
was  apparent  in  several  of  the  Commission  meetings  that  took  place  during  the  RFP  and 
add-back  processes  and  from  the  feedback  given  at  the  focus  group  held  by  the  auditors. 
On  a larger  scale,  the  effect  of  the  funding  decisions  was  that  some  contractors  which 
might  have  been  ill  equipped  to  provide  services  were  funded.  The  impact  on  senior 
services  is  clear  in  the  case  of  the  Commission’s  decision  to  fund  Filipino  American 
Council  for  three  separate  bids  (Community  Services,  Home-Delivered  Meals  (HDM), 
Congregate  Meals)  worth  a total  of  around  $109,000  even  though  the  contractor  scored 
well  below  70  points  on  each  bid.  Another  bidder  scored  and  ranked  higher  in  one  of  the 
categories  but  was  not  awarded  funding.  Filipino  American  Council  was  subsequently 
put  on  probation  for  failing  to  meet  its  obligations  and  later  de-funded  six  months  into  the 
fiscal  year  (January  2004).  The  Department  then  contracted  with  West  Bay  Pilipino  for 
the  remaining  part  of  the  fiscal  year  for  Community  Services  and  Congregate  Meals,  but 
a new  home-delivered  meals  contract  was  not  provided.  In  this  case.  West  Bay  Pilipino 
had  not  originally  submitted  a bid  for  Congregate  Meals  and  thus  did  not  demonstrate  its 
capacity  to  deliver  that  service  through  the  RFP  process,  yet  it  received  a contract  for 
$31,516  following  the  de-funding  of  Filipino  American  Council. 

Most  of  the  Department’s  Funding 
Guidelines  it  Established  for  Selecting 
Contractors  Were  Not  Followed 

As  indicated  earlier,  the  Department  established  the  following  five  funding  guidelines  for 
awarding  monies  to  bidders: 

“To  the  extent  possible,  the  following  recommendation  guidelines  will  be  utilized: 

1 . Proposals  scoring  under  70  points  should  not  be  funded. 

2.  Bids  should  not  be  funded  for  more  dollars  than  requested. 

3.  Services  should  be  available  in  underserved  geographical  areas. 

4.  Services  should  be  assured  for  targeted  populations. 

5.  Awards  should  approximate  current  service.” 

Out  of  the  five  guidelines  established,  the  Department  did  not  follow  three  of  them  and  it 
is  unclear  whether  one  was  followed.  The  first  guideline  states  that  proposals  scoring 
under  70  points  should  not  be  funded.  As  previously  discussed,  this  guideline  was  not 
followed  as  the  Department’s  first  recommendations  to  the  Commission  included  funding 
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eight  bids  that  scored  under  70  points  and  16  out  of  the  20  bids  that  scored  under  70  were 
funded  by  the  end  of  the  add-back  allocation  process. 

The  second  guideline  states  that  bids  should  not  be  funded  for  more  dollars  than 
requested.  However,  the  final  awards  include  1 8 separate  occasions  where  funding 
exceeds  the  bidder’s  request.  The  justifications  for  awarding  more  funding  than 
requested  are  not  based  on  RFP  scores  or  rankings  but  rather  on  separate  deliberations 
made  by  the  Department  and  Commission.  For  example,  for  Community  Services,  two 
bidders — Project  Open  Hand  and  International  Institute  of  San  Francisco  received  more 
funding  than  requested.  None  of  the  19  bidders  that  scored  and  ranked  higher  than  these 
two  bidders  received  more  than  they  requested.  For  Congregate  Meals,  one  bidder — 
Project  Open  Hand,  received  $66,188  more  than  it  requested  even  though  it  was 
providing  less  service  than  the  previous  year.  In  the  same  category,  another  bidder,  Self 
Help  for  the  Elderly,  ranked  higher  yet  received  $40,000  less  than  the  amount  it  requested 
even  though  it  was  expected  to  expand  services.  A third  example  shows  that  for  home- 
delivered  meals,  one  bidder,  Filipino  American  Council,  was  awarded  $7,100  more  than 
requested,  even  though  it  was  the  lowest  ranking  bidder  in  that  category.  All  but  one  of 
the  other  bidders  for  home-delivered  meals  received  less  than  they  requested. 

The  third  recommendation  guideline  states  that  services  should  be  available  in 
underserved  geographical  areas.  However,  the  RFP  scoring  process  did  not  include  any 
mention  or  specific  evaluation  of  underserved  geographical  areas.  The  location  of  the 
contractor  and  the  geographical  need  for  service  were  not  a part  of  the  scoring  process. 

In  addition,  the  targeted  underserved  geographical  areas  were  not  identified  in  the  RFP 
packet.  The  only  program  that  mentions  specific  geographical  areas  in  the  RFP  packet 
given  to  bidders  was  the  Food  Bag  program.  However,  the  areas  mentioned  (Bayview 
Hunters  Point,  Visitacion  Valley,  OMI,  Western  Addition,  Inner  Mission,  Tenderloin, 
Chinatown,  North  & South  of  Market,  and  Sunset)  cover  a large  portion  of  the  City  and 
are  not  specific  enough  to  be  referred  to  as  underserved  geographical  areas.  This 
program  was  less  than  0.25%  of  the  total  RFP  amount  in  FY  2003-04.  It  cannot  be 
determined  if  services  are  available  in  underserved  geographical  areas  because  of  the  lack 
of  available  information  about  underserved  geographical  areas,  the  lack  of  information 
about  all  contractor’s  capacity  to  serve  these  areas,  and  the  lack  of  geographical 
comparisons  of  bidders. 

The  fourth  recommendation  guideline  states  that  services  should  be  assured  for  targeted 
populations.  It  appears  that  the  fourth  recommendation  guideline  was  not  followed 
because  the  Department’s  funding  recommendations  and  the  Commission’s  final 
decisions  diverted  from  the  capacity  of  bidders  to  serve  targeted  populations  as  indicated 
in  the  RFP  score.  Each  bidder’s  RFP  score  already  included  an  evaluation  of  the 
“effectiveness  in  serving  the  OO A target  populations  as  listed  in  the  RFP’s  Targeting 
mandates.”  This  question  was  worth  10%  of  the  bidder’s  overall  RFP  score.  However, 
consideration  for  assuring  services  to  the  targeted  population  became  part  of  the  selection 
criteria  again  when  the  Department  used  it  as  a funding  guideline  independent  of  the  RFP 
score.  As  a result,  a bidder  may  have  been  selected  for  allegedly  targeting  a particular 
community  when  in  fact  another  bidder’s  RFP  score  is  more  indicative  of  its  capacity  to 
serve  the  targeted  populations. 
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The  Only  Guideline  Followed 
Was  Uncompetitive 

The  fifth  recommendation  guideline  was  the  only  guideline  that  appears  to  have  been 
followed.  It  states  that  awards  should  approximate  current  service.  This  guideline, 
interpreted  by  the  Department  as  keeping  existing  contractors,  contradicts  to  some  degree 
the  competitive  intent  of  the  RFP  process.  The  Department  does  not  seem  to  have 
evaluated  whether  other  bidders  could  provide  the  same  services  or  whether  other  bidders 
could  better  serve  clients. 

The  add-back  funding  was  primarily  used  to  fund  contractors  who  were  not  selected 
during  the  RFP  process  but  had  contracts  the  previous  year  (FY  2002-03).  Specifically, 
at  the  end  of  the  Commission  appeals  process,  17  bids  were  not  awarded  funding.  Out  of 
those  1 7 bids,  seven  contractors  had  been  funded  the  previous  year.  By  the  end  of  the 
add-back  process,  the  Department  “restored”  funds  to  all  but  one.  Nine  of  the  ten  other 
bidders  who  did  not  have  a previous  contract  and  were  also  not  awarded  funding  as  a 
result  of  the  RFP  process  were  not  awarded  any  of  the  add-back  monies.  In  addition,  out 
of  all  the  67  bids  that  received  increased  funding  during  the  add-back  process,  only  one 
(Senior  Action  Network,  HICAP)  did  not  have  an  existing  contract  the  previous  year9.  In 
effect,  the  Department  used  the  add-back  funds  to  fund  existing  contractors  even  if  in 
several  cases  that  meant  not  funding  bidders  that  scored  and  ranked  higher  as  a result  of 
the  RFP  process.  This  occurred  in  the  Community  Services,  Care  Management,  and 
Naturalization  Services  categories.  See  Appendix  D for  specific  details. 

The  lack  of  consistent  and  objective  criteria  used  to  select  all  contractors  or  contractor 
funding  amounts  was  apparently  caused  by  the  Department  and  Commission’s 
unwillingness  to  let  the  RFP  scores  and  rankings  drive  the  funding  decisions.  The 
Department  created  new  funding  guidelines  after  the  RFP  was  issued,  and  interpreted  the 
guidelines  in  ways  that  favored  certain  contractors.  As  noted  above,  the  immediate  effect 
of  the  Department  was  widespread  discontent  among  bidders  and  a sense  of  distrust  of 
the  fairness  of  the  RFP  process.  The  various  weaknesses  in  the  RFP  process  affected  the 
outcomes  such  that  there  is  no  assurance  that  the  Department  contracts  for  services  for 
seniors  and  adults  with  disabilities  from  providers  that  are  the  most  qualified. 

RECOMMENDATIONS 

■ The  Department  should  follow  the  rules  it  establishes  in  the  RFP  process,  clearly 
communicate  the  rules  in  the  RFP  packet  given  to  bidders  and  not  change  the 
rules  during  the  process.  Any  funding  decisions  that  deviate  from  the  rules  should 
be  fully  justified,  documented,  and  the  reasoning  behind  those  justifications 
should  be  complete  and  consistently  applied. 

■ The  Department  must  establish  competitive  funding  guidelines  and  ensure  that 
there  are  mechanisms  to  resolve  conflicts  between  funding  guidelines.  Protocols 
and  rules  should  be  established  for  determining  when  one  funding  guideline 
should  receive  priority  over  another.  The  FY  2003-04  RFP  and  subsequent  add- 


9 For  a service  previously  provided  by  the  Department. 
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back  allocation  process  did  not  have  these  protocols,  with  the  result  that 
inconsistent  funding  decisions  were  made  based  on  differing  funding  guidelines. 

■ The  Department  should  communicate  its  funding  guidelines  to  bidders  before  the 
bids  are  evaluated  and  ranked.  With  this  knowledge,  bidders  would  be  able  to 
demonstrate  the  extent  to  which  they  meet  the  funding  guidelines  by  tailoring 
their  proposals  to  prove  that  they  can  meet  those  guidelines.  Bidders  could  design 
their  proposal  to  demonstrate  that  they  can  provide  service  to  a particular  targeted 
community  or  provide  service  more  efficiently. 

■ The  Department  should  follow  the  guidelines  it  establishes,  note  any  deviation 
from  the  guideline  and  ensure  that  any  change  are  made  through  a fair,  open,  and 
competitive  process.  Although  the  Department  may  have  a legitimate  reason  for 
deviating  from  its  guidelines,  the  justification  and  logic  for  funding  particular 
contractors  and  for  granting  particular  amounts  should  be  consistent. 

■ The  Department  must  also  be  clear  on  how  it  plans  to  use  RFP  scores  and 
rankings  and  should  clarify  whether  the  amounts  granted  are  directly  correlated  to 
the  RFP  scores,  or  what  other  considerations  will  apply. 


FINDING  3-B.  LOBBYING  EFFORTS  INFLUENCED  THE  DEPARTMENT 
AND  THE  COMMISSION’S  FUNDING  DECISIONS  AND  THE 
JUSTIFICATIONS  USED  FOR  SELECTING  CERTAIN  CONTRACTORS  WERE 
SOMETIMES  INCONSISTENT  AND/OR  UNSUPPORTED  BY  ANALYSIS. 


The  level  of  review  and  input  to  the  RFP  process  creates  opportunities  for  changes  that 
might  not  be  consistent  with  the  intent  of  a fair  and  competitive  allocation  process. 
During  the  FY  2003-04  RFP  process,  there  were  several  changes  made  to  the 
Department’s  initial  funding  recommendations  that  went  against  the  RFP  rules  and 
funding  guidelines,  and  did  not  follow  any  consistent  criteria.  Although  the 
Department’s  initial  recommendations  also  went  against  the  RFP  rules  and  funding 
guidelines,  the  subsequent  changes  that  occurred  as  a result  of  advocacy  efforts  further 
exacerbated  the  lack  of  consistent  and  objective  criteria  being  used  to  select  bidders.  The 
level  of  input  in  the  RFP  and  add-back  process  also  created  a situation  that  pitted 
contractors  against  each  other  and  encouraged  advocacy  efforts  to  receive  more  funding. 
The  final  allocation  decisions  seemed  to  have  been  influenced  by  the  level  of  input  and 
review. 

A strong  correlation  exists  between  the  final  funding  decisions  and  the  advocacy  efforts 
of  contractors.  For  example,  the  Commission  did  not  approve  funding  for  Filipino 
American  Council  for  a variety  of  services  it  bid  on.  However,  during  the  RFP  and  add- 
back  meetings,  advocates  for  the  bidder  lobbied  on  its  behalf,  speaking  20  times,  more 
than  for  any  other  organization.  Even  though  the  bidder  scored  very  low  on  all  its  bids, 
the  Commission  decided  to  award  them  $20,000  for  Community  Services,  $68,514  for 
Congregate  Meals,  and  $20,559  for  HDM.  In  another  example,  analysis  suggests  that 
increases  in  funding  for  bidders  in  the  Legal  Services  and  District  Wide  Social  Services 
Worker  categories  during  the  add-back  process  were  also  influenced  by  the  number  of 
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times  that  advocates  spoke  on  behalf  of  a contractor.  Appendix  D describes  these 
correlations. 

In  contrast,  Chinese  Newcomer  did  not  have  any  advocates  that  lobbied  on  its  behalf  in 
any  of  the  RFP  or  add-back  meetings  as  shown  in  the  minutes  of  these  meetings.  Chinese 
Newcomer  was  not  funded  for  Naturalization  Services  during  the  add-back  process, 
despite  the  fact  that  they  scored  above  70  points  and  ranked  8th.  However,  within  the 
Naturalization  Services  category,  another  bidder  scored  and  ranked  lower  and  was 
funded.  That  bidder,  La  Raza  Centro  Legal,  had  advocates  speak  on  its  behalf  at  three 
separate  meetings.  Even  though  La  Raza  scored  under  70  points,  ranked  10th,  and  scored 
20  points  lower  than  Chinese  Newcomer,  they  were  awarded  funding. 

The  Department  and  the  Commission  received  numerous  complaints  from  bidders  as  is 
evident  in  the  RFP  meeting  minutes  and  the  large  number  of  appeals  to  the  Commission’s 
decisions.  A Commissioner  even  admonished  the  behavior  of  contract  agencies  during 
the  RFP  and  add-back  process  citing  incidents  of  uncivil  behavior  and  personal  attacks  on 
Commissioners.  Several  contractors  questioned  the  Department’s  decisions  and  were 
appalled  that  even  though  they  had  higher  RFP  scores  and/or  aimed  to  provide  services  to 
distinct  targeted  populations,  they  were  not  recommended  for  funding  or  did  not  receive 
as  much  funding  as  others  that  scored  lower.  This  created  a situation  where  contractors 
publicly  disputed  each  other’s  capacity  to  serve.  The  dissatisfaction  with  the  RFP 
process  was  echoed  during  the  audit  focus  group  where  most  of  the  attendees  indicated 
dissatisfaction  with  the  RFP  and  add-back  allocation  process.  Many  felt  the  funding 
decisions  were  unfair,  unclear,  and  highly  political. 

Both  the  Department  and  the  Commission  appeared  to  have  been  influenced  by  advocacy 
efforts  and  special  considerations  were  given  to  certain  bidders  that  were  not  analyzed  for 
accuracy  and  applied  equally  to  all  bidders  within  the  service  categories.  If  the 
Department  or  the  Commission  make  special  considerations  and  do  not  follow  their  RFP 
rules  and  guidelines,  these  considerations  should  be  thoroughly  analyzed  and  applied 
equally.  Overall,  the  influence  from  various  groups  seems  to  have  resulted  in  many 
funding  decisions  that  were  not  correlated  to  the  contractor’s  RFP  scores.  Although  input 
from  the  public  is  a valuable  component  of  any  RFP  process,  this  input  should  be 
considered  in  support  of,  and  not  in  conflict  with,  the  RFP  scores  and  rankings.  Such  an 
approach  would  protect  the  Department  and  the  Commission  from  appearing  to  make 
arbitrary  decisions  or  decisions  that  favor  a group  for  reasons  not  related  to  its  service 
delivery  capacity. 

RECOMMENDATIONS 

■ The  Department  should  streamline  its  RFP  process  so  that  the  feedback  received 
from  the  Advisory  Council,  Finance  Committee,  Commission  on  Aging,  and  the 
public  is  considered  in  support  of,  not  instead  of,  the  RFP  scores  and  rankings. 
The  feedback  received  during  public  meetings  should  be  compared  to  the  RFP 
analysis,  needs  assessments,  and  priorities  of  the  Department.  This  comparison 
should  be  consistent,  and  should  be  made  for  all  contractors  within  the  specific 
bid  categories. 
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The  Department  should  adhere  to  its  recommendations  unless  substantial 
evidence  is  presented  that  demonstrates  the  need  for  making  a change.  Simply 
stating  that  a bidder  serves  a particular  population  or  stating  that  by  not  funding  a 
particular  bidder  consumers  will  be  impacted  is  not  enough  to  justify  changing 
funding  recommendations.  The  Department  should  provide  evidence  for  making 
funding  changes  and  in  doing  so  should  ensure  that  the  same  considerations  are 
given  to  every  bidder  within  a service  category. 


FINDING  3-C.  THE  MANNER  IN  WHICH  THE  DEPARTMENT  FUNNELS 
CHANGES  IN  FUNDING  TO  CONTRACTORS  IS  SUBJECT  TO  THE  SAME 
WEAKNESSES  AS  THE  RFP  PROCESS. 


The  audit  also  found  that  the  lack  of  adequate  internal  controls  for  determining  contractor 
funding  is  not  limited  to  the  RFP  process.  The  Department  receives  monies  from  the 
State  on  an  ongoing  basis  to  fund  a portion  of  the  services  it  contracts  out.  These  monies 
come  in  at  different  times  during  the  year  and  the  Department  must  make  budgetary 
estimates  about  the  actual  amounts  that  will  be  received.  Oftentimes,  the  State  increases 
funding  for  certain  programs  and  the  Department  thus  has  additional  funds  to  distribute  to 
contractors.  Likewise,  the  State  may  cut  funding  for  particular  programs  during  the  year. 
The  manner  in  which  the  Department  funnels  these  funding  changes  to  contractors  is 
subject  to  the  same  weaknesses  identified  in  the  RFP  process.  Changes  that  occur  as  a 
result  of  more  or  less  money  coming  in  from  the  Federal  and  State  governments  should 
be  allocated  fairly,  openly,  and  competitively.  The  manner  in  which  the  AAA  will  plan 
for  increased  or  decreased  resources  should  also  be  clear  from  the  discussion  of  priorities 
in  the  Department’s  Area  Plan. 

The  Department  Faces  Uncertainty 
Regarding  its  Revenue  Streams 

The  Department  receives  several  million  dollars  in  Federal  and  State  funds  each  year. 
These  funds  are  used  for  programs  supported  by  the  Federal  Older  Americans  Act  and  the 
Older  Californian’s  Act  and  are  earmarked  for  a variety  of  different  services  including 
Congregate  Nutrition,  Home-Delivered  Nutrition,  Medication  Management,  and  several 
other  programs.  The  monies  received  from  the  Federal  and  State  governments  also 
require  a local  match  of  the  City’s  General  Fund  monies.  The  allowable  expenditures  for 
each  of  the  particular  monies  received  are  complicated  and  require  a thorough  knowledge 
of  the  Older  Americans  Act,  Code  of  Federal  Regulations,  California  Code  of 
Regulations,  and  California  Welfare  and  Institutions  Code.  Complicating  matters  further, 
the  amounts  that  the  Department  receives  throughout  the  year  are  subject  to  change  as 
both  the  Federal  and  State  governments  revise  their  funding  amounts.  A portion  of  the 
funding  amounts  that  the  Department  has  to  contract  with  providers  is  thus  always 
pending  availability  of  funds  released  by  the  CDA. 

Further  uncertainty  results  from  the  Department’s  poor  internal  controls  over  revenue  and 
funding  information.  Information  supplied  to  the  auditors  by  Department  staff 
demonstrates  that  the  Department  has  struggled  to  distinguish  between  Federal,  State,  and 
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local  revenues  and  which  monies  were  used  to  fund  contractors  for  specific  programs. 

As  revenues  fluctuate  during  the  year,  it  is  often  not  clear  how  the  Department  converts 
the  changes  in  revenue  to  changes  in  the  contract  amounts  of  specific  providers. 

In  the  past,  the  Department  has  allocated  State  and  Federal  dollars  specifically  to 
individual  contractors  (about  30-35  of  44  Community  Based  Organizations).  This  made 
monthly  and  year-end  reporting  of  State  and  Federal  revenues  (and  accompanying  local 
matches  required)  difficult  and  added  to  the  confusion  regarding  which  contractor  should 
receive  increased  funding  as  additional  dollars  came  in  from  the  State  and  Federal 
governments.  The  Department  admitted  that  the  Office  on  the  Aging  has  not  had  reliable 
contract  information  in  a few  years.  Although  the  Department  has  made  strides  in  its 
administration  of  grants  and  revenues  for  FY  2004-05,  the  Department  struggled  with 
providing  reliable  information  regarding  FY  2003-04. 

For  example,  the  auditors  reviewed  the  final  amounts  paid  to  contractors  for  FY  2003-04 
and  the  contract  amounts  for  FY  2004-05.  The  information  initially  supplied  by  the 
Department  indicated  that  there  were  12  changes  to  contracts  from  the  July  2,  2003  CO  A 
Add-Back  meeting  to  the  Final  FY  2003-04  amounts.  In  addition,  the  initial  information 
supplied  shows  that  were  29  changes  from  the  Final  FY  2003-04  amounts  to  the  FY 
2004-05  contract  amounts.  However,  after  several  exchanges  with  the  Department,  it 
was  determined  that  the  reasons  for  those  changes  had  to  do  with  the  Department 
providing  inaccurate  or  incomplete  information  to  the  auditors,  additional  State  and 
Federal  monies  coming  in,  one  contractor  being  defunded  and  the  remaining  amounts 
being  redistributed,  and  changes  in  accounting  procedures.  The  Department  did  not 
notice  the  discrepancies  until  the  auditors  pointed  them  out.  The  Department  should 
ensure  that  changes  in  contract  amounts  are  continually  monitored  and  that  the  manner  in 
which  the  Department  tracks  funding  per  contractor  is  consistent  from  year-to-year. 

The  Department  Lacks  Standard  Processes 
For  Allocating  Funding  Changes  to  Contractors 

The  uncertainty  surrounding  State  and  Federal  funding  reiterates  the  need  for  strong 
controls  over  how  funds  are  allocated.  Some  of  the  increases  in  funding  from  the  Federal 
and  State  governments  are  earmarked  and  the  Department  is  limited  to  which  services  it 
distributes  the  monies  to.  In  these  cases,  the  Department  does  not  use  a clear  and 
consistent  process  to  allocate  those  increases  to  specific  service  providers.  Likewise, 
when  funding  cuts  must  be  made,  there  is  a lack  of  consistent  approaches  to  determine 
how  cuts  are  made.  Some  of  the  Department’s  recommendations  involve  across-the- 
board  cuts  to  service  providers  while  other  recommendations  involve  targeted  cuts  to 
particular  services.  In  other  cases,  particular  services  are  held  harmless. 

For  example,  DAAS  received  additional  funding  for  the  Family  Caregiver  Support 
Program  during  FY  2003-04.  The  additional  funds  were  distributed  among  the  existing 
five  service  providers  and  each  received  roughly  the  same  percentage  increase  ( 1 7%- 
19%)  to  their  original  contract  amounts.  In  contrast,  funding  changes  that  occurred  for 
IHSS  services  followed  a different  logic.  DAAS  received  a S44,000  workorder  from 
DHS  for  FY  2003-04  to  supplement  its  direct  IHSS  consumer  services.  DAAS  and  the 
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Office  on  the  Aging  had  initially  intended  to  allocate  the  entire  $44,000  for  IHSS 
advocacy  efforts  by  increasing  Planning  for  Elders  in  the  Central  City’s  (PECC)  existing 
contract  of  $45,000.  Upon  reviewing  the  terms  of  the  informal  grant  agreement  with 
DHS,  however,  DAAS  decided  to  reallocate  this  $44,000  workorder  to  other  direct  IHSS 
services. 

In  terms  of  funding  cuts,  the  Department  was  asked  by  the  Mayor’s  Office  to  make 
baseline  and  contingency  reductions  for  FY  2004-05.  The  Department’s  initial  plan  was 
to  first  look  at  actual  RFP  allocations  as  of  the  previous  spring  and  seek  to  reduce  the 
amount  of  additional  funding  that  was  added  via  the  add-back  process  the  previous  year. 
With  a second  plan,  reduction  amounts  would  vary  by  program  according  to  their 
allocated  share  of  the  RFP.  A third  plan  would  grant  a categorical  exemption  for 
nutrition  programs,  home  delivered  meals,  congregate  programs  and  Food  Bag 
programs.  The  ensuing  increase  in  the  community-based  service  reductions  for  all  other 
programs  would  be  16.7%  across  the  board.  After  creating  these  three  plans,  the 
Department  changed  its  strategy  and  allegedly  prioritized  services  and  made 
recommendations  for  cutting  services  based  on  the  Fall  2002  needs  assessment.  In 
particular,  the  Department  recommended  cutting  housing  advocacy  services  and 
naturalization  services.  However,  the  targeted  cuts  recommended  were  not  reflective  of 
the  assessment  as  housing  was  one  of  the  most,  if  not  the  most,  important  need  identified 
from  the  assessment.  The  Department  later  changed  its  reduction  recommendations.  The 
housing  advocacy  and  naturalization  program  cuts  were  replaced  by  across-the-board 
reductions  to  all  programs,  except  in-home  services  and  nutrition  programs  (i.e.,  Food 
Bag,  Congregate  Nutrition  and  Home-Delivered  Meals).  There  was  no  justification 
provided  for  why  in-home  services  and  nutrition  programs  were  not  cut  across-the-board. 
In  fact,  cuts  for  other  programs  included  administrative  cuts,  targeted  cuts,  and  some 
across  the  board  cuts  because  DAAS  wanted  to  maintain  all  service  categories.  After  the 
Department  presented  these  changes  to  the  Commission,  the  Commission  asked  the 
Department  to  clarify  the  logic  used  to  make  those  cuts. 

RECOMMENDATIONS 

■ The  Department  should  ensure  that  its  changes  over  the  administration  of  Federal 
and  State  revenues  and  the  allocations  for  contractors  are  reliable.  The 
Department  appears  to  have  made  some  strides  in  this  area,  but  it  needs  to 
continue  to  track  and  make  more  transparent  the  changes  to  contractor’s  funding 
amounts  as  a result  of  changes  in  revenues. 

■ The  Department  must  create  a policy  document  or  funding  guidelines  that  reflect 
how  changes  will  be  funneled  to  contractors  if  revenue  streams  change.  The  plan 
must  prioritize  services  and  needs  in  a logical  and  competitive  manner — 
providing  more  or  few  units  for  existing  services,  moving  to  a ranked  list  of 
services  that  could  not  be  funded  with  the  original  budget,  or  other  approaches 
supported  by  analysis. 
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CHAPTER  4 

THE  DEPARTMENT  DOES  NOT  HAVE  ADEQUATE 
INFORMATION  ABOUT  THE  OUTCOMES  OF  ITS 
INFORMATION,  REFERRAL  AND  ASSISTANCE  SERV  ICES 


CHAPTER  SUMMARY 

The  Department  funds  I,  R & A services  through  the  ten  Neighborhood  Resource 
Centers  (NRCs),  the  Department’s  internal  I,  R & A unit  and  as  part  of  the 
services  that  some  contractors  must  provide.  Though  the  extent  of  these  services 
and  amount  of  funding  that  goes  into  these  services  is  considerable,  the 
Department  does  not  know  the  results  and  outcomes  of  all  its  I,  R & A services. 
Contractors  who  provide  these  services  do  not  track  them  independent  of  their  other 
services.  The  three  I,  R & A service  providers  aggregate  different  service  data  and  the 
Department  lacks  consistent  definitions  between  the  providers  for  what  should  count  as  a 
unit  of  information,  referral,  follow-up,  or  assistance.  Referral  procedures  between  and 
among  I,  R & A providers  lead  to  duplicate  service  numbers  counted.  As  a result,  the 
Department  cannot  adequately  compare  services  or  assess  the  degree  of  overlap  or 
engage  in  efforts  to  maximize  the  efficiency  of  the  I,  R & A services  it  provides. 


FINDING  4-A.  THE  DEPARTMENT  DOES  NOT  HAVE  ADEQUATE  | 
INFORMATION  ABOUT  THE  RESULTS  AND  OUTCOMES  OF  ITS 
INFORMATION,  REFERRAL  AND  ASSISTANCE  SERVICES. I 


The  Department  of  Aging  and  Adult  Services  (DAAS)  provides  information,  referral  and 
assistance  services  to  seniors  and  adults  with  disabilities  in  three  ways:  the  ten  NRCs  it 
funds,  the  Department’s  internal  I,  R & A unit  and  as  part  of  contracted  services  that 
include  provisions  for  various  I,  R & A related  services.  The  ten  NRCs  are  currently 
operated  by  three  contractors;  the  Institute  on  Aging  (IOA),  Self-Help  for  the  Elderly 
(SHE),  and  Network  For  Elders  (NFE).  Seven  of  these  locations  are  housed  within  a 
center  that  provides  other  direct  services  while  the  other  three  NRCs  are  stand-alone  sites. 
The  Department’s  internal  unit  responds  to  calls  from  the  public  requesting  information 
and  referral,  assistance  and  follow-up  services.  The  Department  also  funds  a number  of 
contractors  that  deliver  I,  R & A related  services  through  a variety  of  disparate  programs 
such  as  the  Housing  Advocacy/Counseling  and  the  Family  Caregiver  Support  Program. 
Because  the  Department  does  not  adequately  track  and  aggregate  the  same  data  among  its 
three  I,  R & A service  providers,  it  is  difficult  to  determine  the  real  scope  of  the 
Department’s  I,  R & A efforts,  compare  the  results  of  the  different  providers,  control 
duplication  of  effort,  or  use  the  information  for  needs  assessment  purposes. 

In  FY  2003-04,  the  Department  spent  at  least  $1,585,000  directly  on  I,  R & A services10, 
not  including  the  funding  for  contractors  that  have  information  and  referral  services  as  an 

10  This  figure  consists  of  5370,000  for  the  estimated  annual  cost  of  the  Department's  internal  I.  R & A unit 
and  the  51,215,000  of  funding  for  the  10  NRCs. 
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ancillary  part  of  their  contract.  This  level  of  funding  makes  I,  R & A the  highest  funded 
service  after  Meals  ($7,852,141)  and  Community  Services  ($2,534,124)  and  means  that 
the  Department  spends  more  money  on  I,  R & A services  than  it  spends  on  almost  all 
direct  service  areas. 

The  Department  Does  not  Adequately 
Track  and  Aggregate  Data  Among  the 
Three  1,  R & A Service  Providers 

Although  the  Department  provides  1,  R & A services  through  a variety  of  means,  the 
providers  are  not  tracking  the  same  information  and  the  Department  is  not  aggregating 
the  results.  For  example,  the  NRCs  track  the  percentage  of  consumers  who  contacted  the 
centers  by  telephoning,  walking-in,  or  other  methods  and  provide  each  center’s  totals  in 
their  annual  report.  However,  the  Department’s  internal  I,  R & A unit  does  not  report 
these  numbers  even  though  their  intake  form  does  track  how  the  contact  occurred. 
Similarly,  the  NRCs  report  a total  of  7,343  follow-up  contacts  and  the  internal  unit 
reports  703,  and  the  intake  form  used  by  both  entities  has  a section  to  track  the  outcome 
of  follow-up  efforts,  but  neither  the  internal  unit  nor  the  NRCs  track  and  aggregate  the 
outcome  of  follow-up  efforts.  The  intake  form  also  has  a section  to  track  the  service 
needs  and  areas  of  clients  that  lists  70  separate  service  needs  and  areas  that  should  be 
tracked  for  each  contact,  but  neither  the  NRC  or  internal  unit  provide  numbers  regarding 
which  service  needs  are  most  requested.  The  internal  unit’s  staff  and  the  NRCs  both 
indicated  that  housing  information  is  the  most  requested  service  but  there  is  no  data 
available  to  determine  the  actual  percentage  of  contacts  that  deal  with  housing. 

information  tracked  by  contractors  providing  I,  R & A related  services  shows  only  broad 
programmatic  categories  and  fails  to  capture  the  amount  of  I,  R & A related  service 
provided.  For  example,  29  organizations  are  funded  by  the  Department  to  provide 
translation  services  as  part  of  their  contract  but  track  only  total  hours  of  translation 
provided  and  not  the  hours  or  number  of  seniors  receiving  I,  R & A related  translation 
assistance.  Similarly,  contractors  are  funded  to  provide  I,  R & A related  services  under 
the  District-Wide  Social  Service  Workers,  Housing  Advocacy/Counseling,  Family 
Caregiver  Support  Program  and  Elder  Abuse  Prevention  Programs.  For  example,  within 
Housing  Advocacy  services,  contractors  are  paid  to  distribute  housing  option  materials 
and  maintain  a current  housing  list  of  affordable  housing  options  and  vacancies  for 
seniors  and  adults  with  disabilities.  Despite  the  fact  that  these  contractors  are  partially 
providing  the  same  services  as  the  NRCs  and  the  Department’s  internal  unit,  there  are  no 
requirements  for  contractors  to  report  their  service  units.  Deficiencies  in  tracking  and 
aggregating  all  of  these  types  of  data  are  summarized  in  Appendix  E. 

The  Department  does  not  have  a consistent  set  of  definitions  or  protocols  that  describe 
how  information  and  referral,  follow-up,  and  assistance  requests  should  be  counted  and 
tracked  by  all  the  I,  R & A service  providers.  The  internal  unit  and  the  NRCs  provided 
some  definitions,  but  their  definitions  are  unclear  on  how  to  distinguish  between 
information  and  referral,  follow-up,  and  assistance  units  of  service.  Moreover,  some 
definitions  of  the  NRCs  and  the  internal  unit  are  not  the  same  and  there  is  some 
indication  that  they  are  counting  the  same  type  of  requests  under  different  categories. 
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For  example,  a person  might  call  an  NRC  to  discuss  a possible  elderly  abuse  case.  Due 
to  the  nature  of  the  inquiry,  the  client  should  be  directed  to  contact  the  Department’s 
Adult  Protective  Services  (APS)  unit.  The  NRC  will  refer  the  caller  to  APS  and  record 
this  contact  as  an  information  and  referral  contact  because  the  NRC  referred  the  client. 
However,  if  the  same  client  called  the  Department’s  internal  unit  instead  of  the  NRC,  the 
caller  would  be  counted  as  unit  of  “assistance”  rather  than  a referral  because  the  internal 
unit  works  closely  with  APS  supervisors  to  assess  the  need  for  APS  involvement. 

As  a result  of  the  lack  of  consistent  definitions  and  a clear  protocols  on  how  to  count  1,  R 
& A units  of  service,  the  Department  is  not  counting  all  units  of  service  provided  in  the 
same  manner  among  its  I,  R & A service  providers  and  the  units  of  service  cannot  be 
compared  because  they  capture  different  types  of  information. 

Referral  Procedures  Among  the 
Various  I,  R & A Service  Providers 
Leads  to  Duplication  of  Service 
Numbers  in  the  Department’s 
Year-End  Totals 

The  practice  of  referring  clients  between  the  NRCs  and  the  Department’s  internal  I,  R & 
A unit  causes  duplication  of  service  numbers.  Department  staff  stated  that  the  internal 
unit  sometimes  receives  calls  from  clients  that  were  referred  to  them  by  the  NRCs. 
Likewise,  Department  staff  stated  that  because  of  their  limited  language  capacity,  they 
sometimes  refer  calls  to  the  NRCs,  which  have  a greater  language  capacity.  The  internal 
unit  does  not  track  how  the  caller  heard  of  them  and  is  thus  unable  to  prevent  double 
counting  of  clients  that  were  referred  to  them  by  the  NRCs.  For  example,  a client  might 
call  an  NRC  to  solicit  information  regarding  Medicare  eligibility  and  be  referred  to  the 
Department’s  internal  unit  that  could  provide  them  with  specific  information.  The  NRC 
would  count  the  contact  as  a referral  contact  while  the  internal  unit  would  count  it  as  a 
“caller.”  These  two  separate  I,  R & A providers  are  essentially  recording  the  same  client 
under  two  different  classifications. 

RECOMMENDATIONS 

■ The  Department  should  first  clearly  define  I,  R & A services  and  consistently 
utilize  one  set  of  definitions  for  all  DAAS  funded  I,  R & A service  providers. 

■ The  Department  should  modify  data  collection  procedures  in  order  to  gather  I,  R 
& A service  data  for  DAAS  funded  contractors  in  enough  detail  to  provide  for 
analysis  and  comparisons  among  the  I,  R & A service  providers. 

■ The  Department’s  internal  I,  R & A unit  and  the  NRCs  should  track  and  aggregate 
data  regarding  the  type  of  information  supplied  to  consumers,  the  outcome  of 
follow-up  activities,  and  the  source  of  all  incoming  referrals.  Simple  referrals 
made  among  the  providers  should  not  be  counted  twice. 
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■ Service  numbers  for  the  three  separate  I,  R & A service  providers  should  be 
combined  and  analyzed  to  determine  the  scope  of  the  Department’s  1,  R & A 
efforts. 


Office  of  the  Controller 


-Page  48- 


May  2,  2005 


APPENDIX  A 


Category 

Dollar  Amount  Allocated 
FY  04-05 

% of  Funds 

Adult  Day  Health/Adult  Day  Care 

$291,209 

1.72% 

Alzheimer’s  Day  Care  Resource  Center 
(ADCRC) 

$293,470 

1.73% 

Brown  Bag 

$50,000 

0.29% 

Care  Management 

$833,052 

4.91% 

Community  Services 

$2,534,124 

14.94% 

Congregate 

$4,292,767 

25.31% 

District  Wide  Social  Service  Wkr  (DWSSW) 

$417,168 

2.46% 

Elder  Abuse  Prevention 

$45,581 

0.27% 

Family  Caregiver  Support  Proqram 

$389,752 

2.30% 

HDM  Clearinghouse 

$75,000 

0.44% 

Health  Insurance  Counseling  and 
Advocacy  Program  (HICAP) 

$182,904 

1.08% 

Health  Screening 

$54,400 

0.32% 

Home-Delivered  Meals  (HDM) 

$3,484,375 

20.55% 

Housing:  Advocacy 

$95,000 

0.56% 

Housing:  Emergency  Assistance 

$40,000 

0.24% 

IHSS  Advocacy 

$45,000 

0.27% 

IHSS  Chore 

$41,600 

0.25% 

IHSS  Homemaker 

$106,800 

0.63% 

IHSS  Personal 

$130,600 

0.77% 

Legal  Services 

$690,000 

4.07% 

Linkages 

$239,249 

1.41% 

Medication  Management 

$18,741 

0.11% 

Naturalization 

$310,000 

1 .83% 

Ombudsman 

$247,862 

1 .46% 

Neighborhood  Resource  Centers  for 
Seniors  and  Adults  with  Disabilities  (NRCs) 

$1,215,000 

7.16% 

Linkages  Service 

$9,449 

0.06% 

Senior  Companion 

$24,585 

0.14% 

Senior  Empowerment 

$108,855 

0.64% 

Transportation 

$691,215 

4.08% 

Total  $16,957,758  100.00% 
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APPENDIX  C 


. — 

Assessment  Type 

Applicable  Program(s) 

Status 

Source  of 
Mandate 

ADCRC  Physical  Plant  Evaluation  Checklist 

ADCRC  programs  (used  for  new  site  only) 

Mandatory 

State 

ADCRC  Self  Assessment  Tool 

ADCRC  programs 

Mandatory 

State 

Annual  Baseline  Assessment 

All  agencies/programs  (except  programs  that 
have  designated  assessment  tools) 

Mandatory 

State 

Annual  Nutrition  Baseline  Assessment 

All  Congregate  and  Home-Delivered  Meals 
programs 

Mandatory 

State 

Annual  Nutrition  Closeout  Summary 

All  Congregate  and  Home-Delivered  Meals 
proqrams 

Mandatory 

State 

Brown  Bag  Program  CDA  5 

Brown  Bag/Food  Bag 

Mandatory 

State 

Caterer's  Assessment 

Congregate  and  Home-Delivered  Meals  that 
have  caterers 

Non-Mandatory 

Local 

Customer  Satisfaction  Survey 

All  aqencies  (composite  summary) 

Mandatory 

Local 

Elderly  Nutrition  Program  Desk  Review  Tool 

All  Congregate  and  Home-Delivered  Meals 
programs 

Non-Mandatory 

State 

Family  Caregiver  Support  Programs  (FCSP) 

FCSP 

Mandatory 

State 

FCSP  Monthly  Expenditure  Report/OOA 
Supplemental  100  Report 

FCSP 

Mandatory 

State 

Financial  Closeout  Report  for  Supportive 
Services 
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Type  of 

Aggregate 

Information 

NRCs 

Internal  Unit 

Contractors  prosiding 
1,  R & A related  services 

Contact  Types 

Aggregate  the  percentages 
of  telephone,  walk-in,  and 
other  types  of  contacts. 

Does  not  aggregate  types 
of  contacts  even  though 
the  intake  form  tracks 
this  information. 

Do  not  track  or  aggregate 
this  information. 

Person  Making 
request 

Aggregate  the  percentages 
of  requests  made  by 
consumers,  family 
members/caregivers,  and 
other. 

Aggregates  the  numbers 
of  callers  that  are  seniors, 
agencies,  caregivers,  and 
other. 

Do  not  track  or  aggregate 
this  information. 

Age  of  person 
making  request 

Aggregate  the  percentages 
of  request  made  by  age 
category. 

Does  not  aggregate  this 
information  even  though 
the  intake  form  tracks 
this  information. 

Do  not  track  or  aggregate 
this  information. 

Type  of  service 
need  and  area  of 
request 

Do  not  aggregate  this 
information  even  though 
the  intake  form  tracks  this 
information. 

Does  not  aggregate  this 
information  even  though 
the  intake  form  tracks 
this  information. 

Do  not  track  or  aggregate 
this  information. 

Outcome  of 
follow-up  efforts 

Do  not  aggregate  this 
information  even  though 
the  intake  form  tracks  this 
information. 

Does  not  aggregate  this 
information  even  though 
the  intake  form  tracks 
this  information. 

Do  not  track  or  aggregate 
this  information. 

Number  of 
information  & 
referral, 
assistance,  and 
follow-up  contacts 

Aggregates  the  numbers  of 
information  and  referral, 
assistance,  and  follow-up 
contacts. 

Aggregates  the  total 
number  of  callers, 
follow-up  calls  made, 
outreach  numbers,  and 
assistance  numbers. 

Do  not  track  or  aggregate 
this  information. 

Number  of 
information  & 
referral, 
assistance,  and 
follow-up  items 
(An  item  is  each 
specific  request 
the  consumer 
makes) 

Aggregates  the  numbers  of 
information  and  referral, 
assistance,  and  follow-up 
items. 

Does  not  track  or 
aggregate  this 
information. 

Do  not  track  or  aggregate 
this  information. 

Total  contacts 

Aggregates  total  contacts. 
(Sum  of  information  & 
referral,  assistance,  and 
follow-up  contacts) 

Aggregates  total  callers. 
(Sum  of  all  calls  from 
seniors,  agencies, 
caregivers,  and  others) 

Do  not  track  or  aggregate 
this  information. 
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City  and  County  of  San  Francisco 


Gavin  Newsom,  Mayor 


Human  Services  Agency 

Department  of  Human  6*tv1om 
Department  of  Aging  and  Adult  Services 

Trent  Nhorar,  Executive  Director 


April  29, 2005 

Ed  Herrington,  Controller 
City  & County  of  San  Francisco 
1 Dr.  Carlton  B.  Goodlett  Place,  #316 
San  Francisco,  CA  94102 

Dear  Mr.  Harrington: 

Thank  you  for  the  opportunity  to  respond  to  your  performance  audit  of  the  Department  of  Aging 
and  Adult  Services  (DAAS),  Audit  Number  04001  P.  We  believe  that  several  of  the  findings  are 
reasonable  and  that  implementation  of  your  recommendations  that  pertain  to  those  findings  will 
improve  the  delivery  of  services  to  seniors  and  adults  with  disabilities  in  San  Francisco. 
Specifically,  we  are  generally  in  agreement  with  the  findings  in  Chapters  2 and  3 of  the  report. 
These  chapters  deal  with  contract  monitoring  and  performance  issues  (Chapter  2)  and  the  RFP 
process  {Chapter  3).  Our  specific  responses  to  these  two  chapters  are  detailed  in  the  following 
sections  of  this  response. 

Chapter  2.  This  chapter  deals  with  contract  monitoring  procedures  and  service  reporting  The 
audit  recommends  that  monitoring  procedures  be  strengthened  and  that  information  reporting  be 
improved.  The  department  has  already  initiated  work  to  replace  the  current  information  system 
(SF  GctCare)  as  recommended  by  the  audit.  In  addition,  based  on  the  audit  findings,  the 
following  steps  will  be  taken: 

• The  vacant  position  of  OOA  Manager  will  be  filled. 

• A written  policies  and  procedures  manual  for  contract  monitoring,  including  fiscal 
monitoring,  will  be  developed.  It  should  be  noted,  however,  that  there  is  an  existing 
policy  on  which  providers  can  perform  self-assessments  based  on  timely  submission  of 
required  units  of  services  and/or  fiscal  reports,  no  change  in  leadership  in  the  previous 
year  and  few  corrective  actions  required  in  the  previous  baseline  assessment. 

• Qualitative  assessment  tools  will  be  developed  to  supplement  existing  program 
monitoring  standards. 

• Program  staff  will  be  trained  in  contract  monitoring  procedures  and  assessment  tools. 

• Program  staff  responsibility  for  contract  monitoring  will  be  rotated  on  a regular  basis. 

• Fiscal  monitoring  forms  and  site  visit  forms  will  be  maintained  in  contract  files. 

Chapter  3.  This  chapter  deals  with  the  RFP  and  contract  award  procedures  followed  by  the 
Department  in  its  most  recent  selection  of  service  providers.  The  audit  recommends  that 
consistent  and  objective  criteria  be  established  to  guide  the  selection  of  service  providers  and  that 
these  criteria  be  followed  in  making  contract  awards.  Based  on  the  audit  findings,  the  following 
steps  will  be  taken: 

• Consistent  and  objective  criteria  will  be  developed  to  guide  the  selection  of  service 
providers  and  the  award  of  contracts. 
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• Selection  criteria  will  be  clearly  stated  in  RFP  packets. 

• Selection  criteria  will  be  followed  by  staff  when  making  award  recommendations. 
However,  it  is  important  to  point  out  our  basic  philosophy  of  providing  a network  of 
accessible  neighborhood  services  that  reflect  the  diversity  of  San  Francisco. 


Although  we  believe  that  the  implementation  of  the  above  recommendations  will  strengthen  the 
provision  of  services  to  seniors  and  adults  with  disabilities  in  San  Francisco,  we  are  concerned 
about  the  limited  scope  of  the  audit.  DAAS  has  undergone  tremendous  changes  over  the  past 
year.  Administrative  staff  was  reduced  significantly  in  the  F Y04-05  budget  and  the  merger  of 
DAAS  into  the  Human  Services  Agency  resulted  in  a complete  reorganization  of  the  support 
services  for  the  Department.  At  the  entrance  conference  with  your  staff,  we  specifically 
requested  your  assessment  of  the  following  areas: 

• The  adequacy  of  administrative  staff. 

• The  adequacy  of  accounting  and  fiscal  staff  given  that  all  of  the  DAAS  finance  and 
accounting  staff  was  eliminated  before  the  merger  of  the  Department  into  the  Human 
Services  Agency. 

• The  appropriateness  of  classifications  used  by  DAAS  for  various  administrative  functions 
including  contract  monitoring,  court  accounting  and  planning. 

Unfortunately,  the  audit  does  not  address  any  of  these  important  areas  of  concern. 

Further,  the  audit  does  not  address  the  changes  that  have  been  undertaken  since  the  merger  of 
DAAS  into  the  Human  Services  Agency.  Chapter  1 addresses  planning  and  needs  assessment 
issues  and  recommends  a number  of  changes  in  the  development  of  the  Area  Plan  that  is  vised  to 
guide  funding  decisions  and  the  allocation  of  OOA  resources  based  on  that  plan.  However,  in 
reaching  its  conclusions  and  recommendations,  the  audit  focuses  on  the  Area  Plan  developed  in 
200 1 and  does  not  acknowledge  the  planning  effort  that  has  been  under  way  since  last  fall  to 
update  the  Area  Plan  as  a guide  for  the  next  four  years.  The  new  Area  Plan  addresses  many  of 
the  recommendations  in  the  audit. 

The  audit  emphasizes  that  nutrition,  the  main  service  funded  through  the  Area  Plan,  was  not  one 
of  the  top  three  needs  cited  in  the  2001  assessment.  However,  nutrition  was  a top  need  identified 
in  the  2002  OOA  assessment  that  shaped  the  current  cycle  of  grants.  The  audit  states  that 
housing,  transportation  and  in-home  support  services  were  the  top  three  priorities.  However, 
home  and  community  based  services,  not  in-home  support  services,  was  one  of  the  top  three 
priorities  and  nutrition  is  included  in  home  and  community  based  services. 

Additionally,  the  audit  recommends  an  unrealistic  process  for  planning.  The  audit  recommends 
a mechanistic  process  in  which  the  universe  of  needs  is  identified  and  ranked,  and  funding  is 
split  in  a proportional  way  based  on  the  assessment  results.  The  audit  emphasizes  that  housing 
was  the  highest  need  cited  in  the  200 1 assessment,  yet  none  of  the  Office  on  the  Aging  (OOA) 
funding  was  directed  at  this  issue.  First,  the  type  of  gap  analysis  recommended  by  the  audit  is 
typically  performed  when  new  funding  is  available  and  communities  want  to  compare  needs  to 
existing  services  and  plug  the  new  funds  into  gaps  in  services.  Almost  15,000  vulnerable  seniors 
depend  on  the  OOA  services  currently  being  funded. . The  2005  gap  analysis  was  conducted 
within  the  context  of  a base  of  existing  services  that  are  in  demonstrable  demand  and  that  are  too 
fundamental  to  the  needs  of  low-income  seniors  to  be  changed  drastically.  Secondly,  a 
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mechanistic  gap  analysis  would  overlook  hidden  populations  like  isolated  and  low-income 
persons  who  are  not  likely  to  participate  in  surveys,  town  halls,  or  focus  groups.  Finally,  the 
entire  OOA  budget  could  not  begin  to  make  a dent  in  the  issue  of  housing  in  San  Francisco, 
which  is  why  it  is  necessary  to  go  through  a public  hearing  process,  as  is  required  by  the 
California  Department  on  Aging,  to  develop  consensus  about  where  OOA  hinds  can  make  a 
difference. 

A similar  problem  of  not  addressing  current  developments  can  be  seen  in  Chapter  4 of  the  audit 
which  deals  with  Information,  Referral  and  Assistance  services.  The  audit  is  critical  of  the 
Department  for  not  having  adequate  information  about  the  results  and  outcomes  of  its 
information,  referral  and  assistance  services.  The  audit  does  not  report  that  the  Department  is 
currently  developing  and  implementing  an  electronic  caller  intake  tracking  system  that  will 
standardize  all  IR&A  tracking  for  DAAS.  This  system  will  track  the  following  types  of 
information: 

• Date 

• Worker  number 

• Source  of  contact  (in-person,  telephone,  outreach,  e-mail) 

• Type  of  contact  (senior,  caregiver,  neighbor,  friends,  community  provider) 

• Services  requested 

• Items  of  assistance 

• Types  of  assistance 

• Client  name 

• Clients  address 

• Client  telephone  number 

• Client  e-mail  address 

• Ethnicity 

• Gender 

• Sexual  Orientation 

• Age 

• Income 

• Disability 

• Living  situation 

• Origin  of  referral 

• Translation  needs 

This  tracking  system  has  been  in  development  for  five  months  with  anticipated  implementation 
in  May  2005.  Development  of  this  electronic  caller  intake  system  has  included  input  from  the 
Office  on  the  Aging  and  staff  from  the  Resource  Centers  for  Seniors  and  Adults  with  Disabilities 
to  incorporate  all  reporting  requirements  and  user  needs.  Once  the  system  is  working  smoothly 
within  DAAS  the  system  will  be  extended  to  the  ten  Resource  Centers  for  Seniors  and  Adults 
with  Disabilities  to  standardize  tracking  and  eliminate  duplication. 

Another  concern  we  have  with  the  audit  is  its  failure  to  place  services  provided  by  the  OOA  in 
the  context  of  services  provided  to  seniors  and  adults  with  disabilities  by  the  rest  of  DAAS  and 
by  the  Department  of  Human  Services.  The  audit  criticizes  the  Department  for  providing  limited 
funding,  through  OOA  contracts,  to  housing,  transportation  and  in-home  support  services. 
However,  as  acknowledged  in  the  audit  report,  within  the  Human  Services  Agency  the  In-Home 
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Support  Services  program  provides  approximately  $75  million  in  services  to  frail  seniors  and 
adults  with  disabilities,  the  Housing  and  Homeless  program  provides  more  than  $40  million  in 
housing  services,  including  services  specifically  targeted  to  seniors,  and  DAAS  provides 
approximately  $700,000  annually  to  fiind  paratransit  programs.  All  of  these  funds  are  provided 
outside  of  the  OOA  contracts  which  comprised  the  limited  scope  of  this  audit. 

In  conclusion,  we  believe  that  the  audit  contains  many  recommendations  for  contract  monitoring 
and  contract  awarding  that  will,  when  implemented,  improve  the  provision  of  services  to  Seniors 
and  Adults  with  Disabilities  in  San  Francisco.  We  also  believe  that  the  audit  would  have 
benefited  from  a review  of  many  of  the  changes  currently  taking  place  in  the  areas  of  planning, 
administrative  consolidation  and  the  development  of  outcome  measures.  We  also  believe  that  an 
expanded  scope  for  this  audit  would  have  provided  a better  context  for  the  funding  allocation 
process  used  for  the  provision  of  services  through  the  OOA  contracts. 

Thank  you  again  for  the  opportunity  to  respond  to  this  audit. 


Sincerely, 


Human  Services  Agency 


Darrick  Lam,  Director 

Department  of  Aging  and  Adult  Services 
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Mayor 

Board  of  Supervisors 
Civil  Grand  Jury 
Public  Library 
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